The actor's agent, Dianne Busch, said that he passed away at his home in South Orange, N.J.
"Jim died from melanoma, which was diagnosed in 1992. He has been fighting it all this time," Busch told ABC News.
The 65-year-old played the father of CIA officer Carrie Matthison on the Showtime drama "Homeland". He also had a recurring role on the USA Network series "White Collar" playing the FBI head of a white-collar crime unit.
Rebhorn also appeared in "Meet the Parents," "Independence Day" and "Scent of a Woman", in which he starred alongside Al Pacino and Philip Seymour Hoffman.
In five decades of television and film work, he also starred in the series finale of "Seinfeld," in which he famously sent the group to jail.
Actress Frances Fisher, who appears on ABC's "Resurrection" tweeted the news Sunday morning: "Shocked and saddened to hear of James Rebhorn's passing. "Jim was a consummate actor with whom I had the pleasure to work with when we were young."
ABC News and The Associated Press contributed to this report.
