In statement released Sunday night, Kevin Quinn, the school's senior vice president for public affairs, said Fine has been "terminated, effective immediately."
The move came at the direction of Chancellor Nancy Cantor.
Two former Syracuse ball boys were the first to accuse Fine, who has called the allegations "patently false."
Earlier Sunday, a third person stepped forward with molestation accusations against Fine.
Zach Tomaselli, 23, of Lewiston, Maine, claims the assistant coach touched him "multiple" times in a Pittsburg hotel room in 2002.
The Post-Standard in Syracuse first reported Tomaselli's accusations. During a telephone interview with The Associated Press, Tomaselli said he signed an affidavit accusing Fine following a meeting with Syracuse police Wednesday.
Tomaselli said the scandal at Penn State involving former assistant football coach Jerry Sandusky prompted him to come forward. Sandusky is accused in a grand jury indictment of sexually abusing eight boys over a 15-year period.
Tomaselli himself is currently facing sexual assault charges in Maine involving a 14-year-old boy.
Syracuse had no choice but to react to the latest revelation from ESPN's "Outside the Lines" investigation. The voice of Fine's wife, Laurie, can be heard on tape discussing what sounds to be incidents of sexual abuse.
The legal telephone recording was made in 2002 by 39-year-old Bobby Davis, who has accused the coach of sexually molesting him for years starting when he was a team ball boy. On the tape, Fine admits she knew her husband had sexually molested Bobby Davis in their home.
In a bizarre twist, Laurie Fine seems to admit that she had a sexual relationship with Davis when he was 18.
Bernie Fine, 65, was in his 36th season at his alma mater. He had the longest active streak of consecutive seasons at one school among assistant coaches in Division I.
Fine maintains he is innocent.
The Associated Press contributed to this report.
