Philadelphia Police Department investigators described the incident as an act of vandalism at the Mt. Carmel Cemetery in the city's Wissinoming district.
Aaron Mallin of North Jersey made the disturbing discovery Sunday when he came to visit his father's grave.
"It's just very disheartening that such a thing would take place," Mallin said.
He said that while the results were disturbing, he was hoping that somehow it was not an anti-Semitic attack.
"I'm hoping it was maybe just some drunk kids. But the fact that there's so many, it leads one to think it could have been targeted," Mallin said.
The damage was widespread throughout the cemetery, with dozens, perhaps hundreds, of headstones affected.
Mallin said he did not understand how someone could do such a thing.
"It's just very heartbreaking," Mallin said.
Israeli Foreign Ministry Spokesman Emmanuel Nahshon released a statement on Twitter saying: "#Philadelphia Jewish cemetery desecration is shocking and a source of worry. Full confidence #US authorities catch and punish culprits."
Vandals target Orchard Park cars, bridges with Nazi and racist graffiti in Buffalo. Evil and hatred must be stopped. Now.— Emmanuel Nahshon (@EmmanuelNahshon) February 26, 2017
Jewish Federation of Greater Philadelphia officials told WPVI-TV that will be opening a mailbox at jewishphilly.org later Sunday to begin raising money to help speed up the repairs at the cemetery.