Zapatero also withdrew Spanish troops from Iraq and launched a drive to cede more power to Spain's semiautonomous regions.
"The Spanish people have spoken clearly and decided to start a new era," Zapatero told euphoric supporters outside the party's headquarters in Madrid. "I will govern with a firm but open hand."
With 97 percent of the vote counted, Zapatero's Socialist party had 43.7 percent, versus 40.1 percent for the conservative Popular Party, according to the Interior Ministry.
Opposition conservatives conceded defeat, but took solace in picking up more parliamentary seats than the Socialists. Both parties gained seats at the expense of smaller leftist and regional groups.
While Zapatero's party picked up seats in the lower house, it fell short of a majority and will have to form an alliance with smaller regional parties to govern.
For Mariano Rajoy, Zapatero's conservative rival both in Sunday's vote and in the 2004 election, his second defeat will likely increase pressure on him to step down as party chief.
"I thought Rajoy would do better, he speaks with such conviction I'm surprised he fell behind," said Jose Eguren, a 29-year-old security guard in the Basque city of Bilbao.
Sunday's vote came two days after the killing of Socialist politician Isaias Carrasco at the hands of the Basque militant group ETA, which jolted Spaniards and prompted both parties to cancel final campaign appearances.
Some in Spain had predicted the killing might prompt a wave of sympathy and boost at the polls for Zapatero's party, especially after Carrasco's 20-year-old daughter Sandra made an emotional appeal Saturday for people to defy ETA by turning out en masse to vote.
The timing of the attack was reminiscent of the massacre by Islamic militants who killed 191 people in bombings against commuter trains in Madrid on March 11, 2004.
Three days after that attack, Zapatero won a surprise victory amid a wave of voter outrage at the ruling conservatives, who blamed them on ETA even as evidence of Islamic involvement mounted. The tactics were widely seen as a bid to deflect perceptions that the killings were al-Qaida's revenge for the government's deeply unpopular support of the U.S.-led invasion of Iraq.
Many conservatives considered Zapatero's 2004 victory a fluke, and saw Sunday's vote as their chance to correct it. The prime minister's victory was seen as finally giving him a legitimacy that critics say he has lacked.
Sunday's results showed the Socialists on their way to winning 168 seats in the 350-seat lower house, up from the current 164 but shy of the 176 seats needed for an outright majority.
The Popular Party was also shown picking up seats, raising its total from 148 to 154.
In his next term, Zapatero's main task will be to reboot the once booming but now slowing economy, shaken by the subprime mortgage crisis in the U.S. and a cooling construction sector.
Zapatero also faces the challenge of the resurgence of ETA, which ended a cease-fire in December 2006 and made a dramatic show of force with the killing of Carrasco.
The campaign was marked by acrimony, with Rajoy hammering Zapatero on everything from immigration to the economy.
In two televised debates between the men, Rajoy used a form of the word "liar" more than 30 times to describe Zapatero. He also blamed Zapatero for not doing enough to spur the cooling economy.
Rajoy vowed to make immigrants sign a contract obliging them to respect Spanish customs and learn the language, a position Zapatero's party called xenophobic. The candidates also clashed on Zapatero's willingness to grant more self-rule to Spain's semiautonomous regions, which conservatives warn will tear apart the nation.
All of these issues have left Spain polarized. Enrique Monreal, 35, a publishing company employee, expects the confrontational climate to continue.
"It will take several years for things to calm down. Right now it is so tense you are nervous even talking to your neighbor," Monreal said outside a polling station in Madrid.