Investigators said a gravel truck rear-ended the BMW on U.S. Route 101 before plunging into a creek, killing the trucker.
The BMW went halfway through a railing, leaving part of the car dangling over the 100-foot drop.
Firefighters used a forklift to hold up the car while the woman and her two daughters, ages 10 and 10 months, were pulled out.
The San Juan Capistrano family was safely taken out of their demolished car an hour later. They were all hospitalized in serious condition.
The trucker died when his rig crashed into the creek and burst into flames.
The Associated Press contributed to this report.