The faithful were eager to join the new pope at the start of solemn Holy Week ceremonies, which lead up to Easter, Christianity's most important day.
Keeping with his spontaneous style, the pope broke away several times from the text of his prepared homily to encourage the faithful to lead simple lives.
Francis also encouraged people to be humble and stay young at heart as the first pope from Latin America promised to go to a youth jamboree in Brazil this summer.
Following the two-hour Mass, Francis boarded the open-topped popemobile to circle through the enthusiastic crowd. He leaned out to shake hands, kissed and patted the heads of infants passed to him by bodyguards, and often gave children the thumbs-up sign.
The pope even climbed down from the vehicle, kissed a woman in the crowd and chatted briefly with her, and another man in the crowd leaned over a barrier to squeeze the pontiff on a shoulder - an unheard of familiarity in the previous pontificate of the reserved Benedict XVI.
Palm Sunday recalls Jesus' entry into Jerusalem but its Gospel also recounts how he was betrayed by one of his apostles and ultimately sentenced to death on a cross.
Recalling the triumphant welcome into Jerusalem, Francis said Jesus "awakened so many hopes in the heart, above all among humble, simple, poor, forgotten people, those who don't matter in the eyes of the world."
Francis then told an off-the-cuff story from his childhood in Argentina. "My grandmother used to say, 'children, burial shrouds don't have' pockets'" the pope said, in a variation of "you can't take it with you."
Since his election on March 13, Francis has put the downtrodden and poor at the center of his mission as pope, keeping with the priorities of his Jesuit tradition. His name - the first time a pope has called himself 'Francis' - is inspired by St. Francis of Assisi, who renounced a life of high-living for austere poverty and simplicity to preach Jesus' message to the poor.
Francis presided over the Mass at an altar sheltered by a white canopy on the steps of St. Peter's Basilica.
The Associated Press contributed to this report.
