The FBI said Hafiz Khan, 76, who is the spiritual leader at a mosque in Miami, was taken into custody in Florida.
Two of his adult sons are also in custody.
One of the sons, 37-year-old Miami limo driver Irfan Khan, was detained in a Los Angeles hotel. Authorities didn't say what he was doing in Southern California.
The other son, 24-year-old Izhar Khan, who is also an imam, was arrested in Florida.
The men are U.S. citizens. Their mosques are not suspected of wrongdoing, officials said.
An adult daughter, a younger grandson and an unrelated man in Pakistan were also expected to be charged. Authorities said they are hoping the three suspects in Pakistan will soon be in custody and will ultimately be extradited to the U.S. to stand trial.
Federal authorities said the group supported terrorists in furthering acts of murder, maiming and kidnapping.
They're also accused of providing some $50,000 to finance the Taliban in Pakistan to buy guns and train young jihadist.
"The motive for sending the funds and facilitating the Pakistani Taliban is in order to make sure the terrorist organization continues its mission," U.S. attorney Wilfredo Ferrer said.
Investigators said the trio's main target were U.S. troops, which have been under attack by Taliban forces.
According to the indictment, Hafiz Khan had a conversation with a Taliban contact in which he hoped for an attack that would kill 50,000 or more American soldiers.
Another of Khan's sons told a local newspaper no one in his family supports the Taliban.
Several arraignments are set for next week. If convicted of the federal charges, the father and sons could face up to 15 years in prison.
The Associated Press contributed to this report.
