The blaze broke out Thursday at about 4:30 p.m. at a home on the 800 block of East Lennon Street in West Rancho Dominguez.
Officials said the mother and son were upstairs, and at some point, the boy went downstairs.
The mother smelled smoke and tried to go downstairs, but the fire was too strong. She escaped by climbing down from a second-story balcony. The boy died at a hospital.
Thursday night, neighbors prayed for the family.
"My heart just goes out to the family members and everybody. We are all grieving right now in our neighborhood," said neighbor Arnold Webb.
Neighbors said the mother lived at the home with her three children: the 3-year-old who died, a 9-year-old son and a 20-year-old daughter.
Authorities said the daughter was not home when the fire broke out, but she and an adult male were driven away from the scene Thursday in the back of a sheriff's patrol car.
There was no word on how the fire may have started.