Beth Humphrey, 30, who is white, and 32-year-old Terrence McKay, who is black, say they're contacting the U.S. Justice Department about filing a discrimination complaint.
"That was one thing that made this so unbelievable," Humphrey said. "It's not something you expect in this day and age."
Bardwell claims he's not a racist.
"There is a problem with both groups accepting a child from such a marriage," he said. "I think those children suffer and I won't help put them through it."
"I have piles and piles of black friends," he adds. "They come to my home, I marry them, they use my bathroom. I treat them just like everyone else."
Bardwell estimates he has refused to marry four interracial couples during his career.
"It is really astonishing and disappointing to see this come up in 2009," said American Civil Liberties Union of Louisiana attorney Katie Schwartzmann. She said the Supreme Court ruled in 1967, "That the government cannot tell people who they can and cannot marry."
The ACLU has asked the Louisiana Judiciary Committee to investigate Bardwell, calling for severe sanctions against him.
Meanwhile, newlyweds Humphrey and McKay found another judge to marry them.
The Associated Press contributed to this report.