Zimmer had one of the longest-lasting careers in baseball history. He played for the Dodgers, Mets, Cubs, Cincinnati Reds and Washington Senators, and he also managed San Diego, Boston, Texas and the Chicago Cubs.
"Great baseball man. A baseball lifer. Was a mentor to me," Yankees manager Joe Girardi said.
During the Yankees' latest dynasty, Zimmer was Joe Torre's right-hand man as the bench coach.
"Zim was around when I first came up. He was someone that taught me a lot about the game -- he's been around, he's pretty much seen everything," Jeter said.
Zimmer had been in a rehabilitation center since having seven hours of heart surgery in April. He was still working for the Tampa Bay Rays as a senior adviser when he died at a hospital in Dunedin, Fla.
"Today we all lost a national treasure and a wonderful man," Rays principal owner Stuart Sternberg said in a statement.
Before Wednesday night's game at Dodger Stadium, a moment of silence was held to remember the man who played alongside Jackie Robinson.
Zimmer is survived by his wife; son Thomas, a scout with the San Francisco Giants; daughter Donna, and four grandchildren.
The Associated Press contributed to this report.