The disaster killed just over 19,000 people and triggered the world's worst nuclear crisis in a quarter century.
In the battered northeast coastal town of Rikuzentakata, a siren sounded at 2:46 p.m. - the exact time the magnitude-9.0 quake struck on March 11, 2011. Also, a Buddhist priest rang a huge bell at a temple overlooking a barren area where homes once stood.
At the same time in the seaside town of Onagawa, people facing the sea pressed their hands together in silent prayer.
Meanwhile, at a memorial service in Tokyo's National Theater, 78-year-old Emperor Akihito, Empress Michiko and Prime Minister Yoshihiko Noda stood in silence with hundreds of other people dressed in black.
Even in Tokyo's busy shopping district of Shibuya, pedestrians briefly stopped and fell silent before carrying on.
Members of Los Angeles' Japanese community observed the one year anniversary of the Japan earthquake and tsunami as well.
The Nishi Hongwanji buddhist temple in Little Tokyo held a special memorial service. They gathered to honor the lives lost and to pray for those still missing.
Some who gathered for the service have witnessed the devastation first hand, while others had family members caught in the disaster.
"My brother was in Sendai," said Mike Okamoto, who attended the ceremony. "Luckily he was OK, but we were not able to get in touch with him for about two weeks, so we worried."
Many of those in attendance said they were grateful the world has not forgotten the destruction that took place in Japan.
The prime minister of Japan said in a speech that the nation's people have overcome disasters and difficulties many times in the past, and pledged to rebuild the nation and the area around the tsunami-stricken Fukushima nuclear plant so that the country will be "reborn as an even better place."
"Our predecessors who brought prosperity to Japan have repeatedly risen up from crises, every time becoming stronger," Noda said. "We will stand by the people from the disaster-hit areas and join hands to achieve the historic task of rebuilding."
The earthquake was the strongest recorded in the history of Japan and set off a tsunami that swelled to more than 65 feet in some areas along the northeastern coast, bringing widespread destruction.
The tsunami also knocked out the cooling systems at the Fukushima Dai-ichi nuclear power plant, causing meltdowns at three reactors and spewing radiation into the air. Some 100,000 residents who were forced to flee the area remain in temporary housing or with relatives, and a 12-mile area around the plant is still off limits.
All told, some 325,000 people rendered homeless or evacuated are still in temporary housing. While much of the debris along the tsunami-ravaged coast has been gathered into massive piles, very little rebuilding has begun.
In addition to the massive cleanup, many towns are still finalizing reconstruction plans, some of which involve ambitious and costly projects of moving residential areas to higher and safer ground. Bureaucratic delays in coordination between the central government and local officials have slowed rebuilding efforts.
Protesters in Tokyo also held a moment of silence on Sunday before marching toward the headquarters of Tokyo Electric Power Co., the operator of the Fukushima nuclear power plant. Public opposition to nuclear power has grown in the wake of the disaster, the worst since Chernobyl in 1986.
The government says the Fukushima plant is basically stable and radiation has subsided significantly, but the plant's chief acknowledged to journalists visiting the complex recently that it remains in a fragile state, and makeshift equipment - some mended with tape - could be seen keeping crucial systems running.
Meantime, six months after experiencing the massive earthquake first hand, a group of Southland performers returned to Japan to give back to the tsunami victims.
Forty-five members of The Young Americans music outreach group were touring near Tokyo when the quake hit. They were nowhere near the tsunami, but it still took them about a week to get home.
The group returned to Japan last fall. This time, they brought their program to areas wiped out by the tsunami.
"It's a 6-year-old child that subliminally he knows that one of his family members is gone. His whole entire home is gone. His parents' income is gone and he's a little kid and it's just weighing so hard on them and all you want to do is give them the chance to be a child and be happy again," described Mary Leist, part of The Young Americans.
The performers say they were shocked to see so much devastation, but felt blessed to bring joy to the survivors who were still struggling to rebuild their lives.
The Associated Press contributed to this report.