Rice was let go a day after video aired on ESPN showing the coach kicking and shoving players and shouting gay slurs at them during practice.
The video drew outrage on social media as well as criticism from state officials, including New Jersey Gov. Chris Christie.
The university's athletic director, Tim Pernetti, was shown the video in late November, but only decided to suspend Rice for three games, fine him $50,000 and order him to attend anger management classes. Rutgers' president signed off on the initial punishment.
But Rutgers officials on Wednesday referred to new information and "a review of previously discovered issues" as the reasons for Rice's termination.
"I am responsible for the decision to attempt a rehabilitation of Coach Rice," Pernetti said. "Dismissal and corrective action were debated in December and I thought it was in the best interest of everyone to rehabilitate, but I was wrong. Moving forward, I will work to regain the trust of the Rutgers community."
The Associated Press contributed to this report.
