The collection program was disclosed by NSA leaker Edward Snowden, a former systems analyst who released thousands of documents that exposed details of U.S. surveillance programs.
U.S. District Court Judge Richard Leon granted a preliminary injunction sought by plaintiffs Larry Klayman and Charles Strange, concluding they were likely to prevail in their constitutional challenge.
"I believe that the bulk telephony metadata collection and analysis almost certainly does violate a reasonable expectation of privacy," Leon said in his ruling.
Leon, an appointee of former President George W. Bush, ruled that the two men are likely to be able to show that their privacy interests outweigh the government's interest in collecting the data. Leon says that means that the massive collection program is an unreasonable search under the Constitution's Fourth Amendment.
"Given the ... utter lack of evidence that a terrorist attack has ever been prevented because searching the NSA database was faster than other investigative tactics -- I have serious doubts about the efficacy of the metadata collection program as a means of conducting time-sensitive investigations in cases involving imminent threats of terrorism," Leon wrote in his 68-page opinion.
The Obama administration has defended the program as a crucial tool against terrorism.
The Associated Press contributed to this report.