The justices indicated they will invalidate part of DOMA if they can get past procedural problems similar to those that appeared to mark Tuesday's case over California's ban on same-sex marriage.
Justice Anthony Kennedy, often the decisive vote in close cases, joined the four more liberal justices in raising questions about part of the DOMA that prevents married gay couples from receiving benefits afforded to straight married Americans.
Kennedy said the law seems to intrude on the power of states that recognize same-sex marriages. Other justices said the law creates what Justice Ruth Bader Ginsburg called two classes of marriage, full and "skim-milk marriage."
The federal law affects a range of benefits available to married couples, including tax breaks, survivor benefits and health insurance for spouses of federal employees.
Some of the more conservative justices were worried about the unintended consequences of overturning DOMA.
Lower federal courts have struck down the measure, and now the justices will have to decide whether to follow suit - asking the question: Are married same-sex couples entitled to those federal benefits?
Justice Elena Kagan repeatedly questioned the motivation behind DOMA, which was passed by large majorities in Congress and signed by President Bill Clinton.
She read from a House of Representatives report explaining that the reason for the law was "to express moral disapproval of homosexuality." The quote produced an audible reaction in the courtroom.
Paul Clement, representing the House Republican leadership in defending the law, said the more relevant question is whether Congress had "any rational basis for the statute." He supplied one, the federal government's interest in treating same-sex couples the same no matter where they live.
Opposing Clement was the Obama administration's top Supreme Court lawyer, Donald Verrilli, who said the provision of DOMA at issue, Section 3, impermissibly discriminates against gay people.
Edith Windsor, who was married to her same-sex partner Thea Spyer in 2007, brought the case before the Supreme Court when she was socked with a $363,000 estate tax after Spyer passed away.
There is no dispute that if Windsor had been married to a man, her estate tax bill would have been zero. The U.S. 2nd Circuit Court of Appeals in New York agreed with a district judge that the provision of DOMA deprived Windsor of the constitutional guarantee of equal protection of the law.
"Today is like a spectacular event for me, it's a lifetime kind of event," said the 83-year-old Windsor outside the Supreme Court building Wednesday."I know that the spirit of my late spouse, Thea Spyer, is right here watching and listening and we'd be very proud and happy of where we've come to."
Like the Proposition 8 case from California, Windsor's lawsuit could falter on a legal technicality without a definitive ruling from the high court.
Same-sex marriage is legal in nine states and was also legal in California for less than five months in 2008. If the Supreme Court strikes down the challenged part of the DOMA, then married same-sex couples in those places will start to receive federal benefits.
This case comes a day after the Supreme Court heard arguments on California's Proposition 8, the ban on same-sex marriage. Prop. 8 was passed by voters in 2008, but lawyers representing a gay couple challenging the ban argue that the constitution guarantees equal protection under the law and that marriage is a fundamental right.
The nine justices questioned lawyers from both sides in the hearing. Several justices, including some liberals who seemed open to gay marriage, raised doubts during the riveting 80-minute argument before them.
Kennedy suggested the justices could dismiss the case with no ruling at all. Such an outcome would almost certainly allow gay marriages to resume in California but would have no impact elsewhere.
There was no majority apparent for any particular outcome. The ruling on California's ban on same-sex marriage is not expected until late June.
Outside the courtroom, a somewhat smaller crowd gathered on Wednesday, mainly gay marriage supporters who held American and rainbow flags.
The Associated Press contributed to this report.