The FBI received information from a one-time mafia member that Hoffa's remains were buried in a rural area called Oakland Township north of Detroit.
Hoffa went missing in 1975. He was supposed to meet with two mafia bigwigs in Detroit but went missing.
Various searches throughout the years have turned up nothing. Hoffa's remains have been said to be encased in concrete under Giants Stadium in New Jersey, among other suppositions.
About 40 FBI agents searched a field Monday and Tuesday before calling off the search Wednesday morning. A backhoe was used to dig up ground, and a small field and a gravel road were extensively searched.
The FBI was acting on information supplied by ex-Mafia captain Tony Zerilli, who claimed that once he was out of prison, associates shared information on Hoffa's whereabouts. It's widely believed Hoffa ran afoul of the mafia. He was convicted in 1964 for jury tampering and had links to organized crime.
In previous years, a swimming pool and a horse farm were dug up following leads.
The makeshift grave FBI agents were seeking this week was in a barn beneath a concrete slab in the Oakland Township field, according to Zerilli. The barn is gone, but the slab remains.
Detroit FBI chief Robert Foley said agents pored over about an acre in their 2-and-a-half-day search. The effort included help from forensic anthropologists at Michigan State University and Michigan State Police cadaver-sniffing dogs.
"We did not uncover any evidence relevant to the investigation on James Hoffa," Foley said. "I am very confident of our result here after two-days-plus of diligent effort."
Hoffa's son, James P. Hoffa, is the current Teamsters president, and union spokeswoman Leigh Strope said family members "do hold out hope that they will one day learn what happened."
"We're always hopeful that we'll get a lead that will lead us to a position in which we can conclude this investigation," said Foley, "both for the process of justice but also for the family."
The Associated Press contributed to this report.