The fire was reported shortly before 5 p.m. Sunday in the 6000 block of Sultana Avenue, according to the Los Angeles County Fire Department. Firefighters had the flames out in less than half an hour, but ultimately found three male bodies inside the residence.
It wasn't immediately clear if they died from the fire or another cause.
Two other men who were encountered outside the home were hospitalized and treated for burns to their arms.
Lt. Hugo Reynaga with the Los Angeles County Sheriff's Department homicide unit said another person who lives at the home approached deputies.
"There was a language barrier," Reynaga said. "He speaks Mandarin. At this point he is a person of interest. They do believe he was the person who set the fire."
The name of the 64-year-old man has not been released. No arrests have been made.
It appears the people living at the home were roommates but not necessarily family members.