The court said the part of SB 1070 that requires police to check the status of someone they suspect is not in the country legally could go forward, but the justices said the provision could be subject to additional legal challenges.
Justice Anthony Kennedy said the policy could interfere with federal immigration law, but that the court couldn't assume that it would.
"The Federal Government has brought suit against a sovereign State to challenge the provision even before the law has gone into effect," Kennedy wrote. "There is a basic uncertainty about what the law means and how it will be enforced."
SB 1070 was signed into law by Gov. Jan Brewer in April 2010, but it was immediately challenged by the Obama administration. A lower court sided with the administration and agreed to prevent four of the most controversial provisions from going into effect.
In a statement, Brewer claimed victory because the provision that police can check for papers was upheld.
"Today's decision by the U.S. Supreme Court is a victory for the rule of law. It is also a victory for the 10th Amendment and all Americans who believe in the inherent right and responsibility of states to defend their citizens," Brewer said. "After more than two years of legal challenges, the heart of SB 1070 can now be implemented in accordance with the U.S. Constitution."
Lawmakers and civil rights groups argue the provision was an invitation to racial profiling, and the president was of the same sentiment. The high court's decision comes days after the Obama administration issued a directive that protects younger immigrants who came illegally to the U.S. as children from deportation.
"I am pleased that the Supreme Court has struck down key provisions of Arizona's immigration law. What this decision makes unmistakably clear is that Congress must act on comprehensive immigration reform," President Barack Obama said in a statement.
But he went on to say that he was concerned about the remaining upheld provision.
"No American should ever live under a cloud of suspicion just because of what they look like. Going forward, we must ensure that Arizona law enforcement officials do not enforce this law in a manner that undermines the civil rights of Americans, as the Court's decision recognizes," Mr. Obama said.
The Justice Department has set up a hotline for the public to report potential civil rights concerns regarding the Arizona law that requires police to check the immigration status of those they stop for other reasons.
The hotline phone number is 1-855-353-1010. The email is: SB1070@usdoj.gov.
ABC News and The Associated Press contributed to this report.
