The posters - which included Nazi symbols, anti-Semitic speech and racial slurs against blacks and other minorities - were visible from Park Sorrento, putting residents on edge.
A man who is believed to be the long-time owner of the condo was seen last week yelling and cursing at people from his balcony. No threats were made, according to the sheriff's department, but some neighbors say they feel unsafe and intimidated by the messages.
"I'm horrified, and I'm sad," resident Sheri Polk said. "Sad that people feel this way about Jews, about African Americans."
The city of Calabasas said the material was taken down Friday, the morning after a deadline was issued for the signs' removal. The resident initially refused to remove the posters, according to Calabasas Mayor Alicia Weintraub.
Condo board members say the man has been fined but continues to re-post the signs.
"We will not stand for this type of material being displayed for everybody to see," Weintraub said at a press conference Wednesday.
Officials believe mental illness may have played a factor.
One neighbor said this isn't the first incident involving the condo resident.
"I have seen him around, he comes out screaming, yelling. I don't pay much attention to it," resident Rick Ricco said.
It is unclear what additional action, if any, will be taken against the resident.