The mud and debris that came down Monday is about 8 feet deep and 250 feet long, and it moved a guest house off its foundation. That structure was red tagged, meaning it has been deemed too dangerous to enter.
More: List of storm-related evacuations
The adjacent main home was also damaged. Fortunately, the people who live there were not injured.
MORE: Encino guest house pushed off foundation by heavy debris flow
Now as more rain continues to fall, firefighters fear that the situation could get worse.
"We want them to be voluntarily evacuated. They're not under threat at this moment. It's static - the movement of the hillside, we're not seeing it, but there is a potential," said Erik Scott with LAFD.
One resident, who has lived in the area for 20 years, says she is worried that her home could incur damage from the rain.
"That can be replaced, this cannot," said Susan Richman as she held her dog in her arms. "A life is more important. All the neighbors have been alerted. We are all just watching and waiting, and we are all packed and ready to go on the street here."
Firefighters and city officials were worried about the steep slope sliding more, and so is the wife of the late sportscaster Tom Kelly. She's also under the voluntary evacuation order.
"I'm very much concerned about the front, " she said. "(There) is only one way we can come to our houses... We never know what can happen."
Fire officials said an assessment of the hillside continues as more rain is in the forecast on Wednesday.