The San Jose Mercury News is reporting Orville Brown, 46, of San Jose turned up the music in the vehicle so his passengers couldn't hear him arguing on the phone with his estranged wife.
"The music was really loud. And I kept yelling, 'I can't hear you. Turn it down,'" Rachel Hernandez-Brown told the newspaper. "I said, 'You're not paying attention.' You know, like, get off the phone. Stop calling me."
This raises questions about whether Brown was too distracted to react quickly enough when one of the nine passengers began banging on the closed partition to warn him smoke was filling the limo.
Brown told authorities that he initially misunderstood the warning as a request to smoke a cigarette and kept driving.
Five women were killed when the limousine burst into flames on the San Mateo-Hayward Bridge on May 4. The women were trapped inside the 1999 Lincoln Town Car.
Four other passengers managed to get out of the vehicle after it came to a stop but sustained injuries. Brown was the only person who escaped unhurt.
Hernandez-Brown said Brown called moments after getting out of the limo to tell her it was ablaze.
"He was continuously calling me back," she told the Mercury News. "I said, 'Well, what made you call me first?' He said, 'Well, I don't know, I didn't know who else to call.'"
The couple have four children and separated about a month before the fire on the San Mateo Bridge. Hernandez-Brown called police hours before the blaze to report that Brown had kicked and dented her car during an argument. He had left the scene before police arrived.
Brown has declined to comment on the new allegations.
The Associated Press contributed to this report.