According to a transcript posted by a blogger, the driver called a dispatcher and said, "It wasn't just crying, it was screaming. I just can't drive with it screaming."
Over the loudspeaker, the female bus driver told the woman to distract the baby to quiet it down, passenger Jennifer Chapman.
Passengers were so upset when she booted them from the bus - in the dark no less - that they decided to get off too.
"Some people around me were saying, 'It's not bothering us, why is it bothering you?' And I spoke up to the driver that 'She's a baby, babies don't work that way. You can't just turn them off,'" Chapman said.
Chapman said the child was younger than 2 and was fussing around in her mother's lap at the time. The woman spoke only Spanish, she said.
Under the transit agency's policy, the driver should have called a supervisor. Mothers with children are among those riders the agency says should not be immediately thrown off a bus if there's a problem.
The driver is now on administrative leave.
The Associated Press contributed to this report.