Officials from the U.S. Coast Guard and the FAA said two planes went down around 3:30 p.m. Friday near the Angeles Gate Lighthouse.
Around 2 p.m. Sunday, authorities recovered the wreckage of one of the planes and the remains of two victims about two miles outside of the Los Angeles and Long Beach Ports.
Dive teams found the remains of two of the victims about 105 feet deep in the water. The search efforts were concluded for the day, but are expected to resume Monday morning.
Two men, ages 61 and 81, were in one of the planes together. A 72-year-old woman was the sole occupant of the other plane, according to the Coast Guard.
On Saturday, the search for the victims turned into a recovery effort and crews worked day and night to gather up debris.
The Los Angeles County Department of Medical Examiner-Coroner will work to identify the victims, authorities said.