A national moment of silence was held at 2:46 p.m. (0546 UTC), the time when the magnitude 9 earthquake struck off Japan's coast three years ago. Tsunami alarm sirens were also sounded to mark the disaster.
Emperor Akihito paid tribute to victims killed in the tragedy and those struggling in its aftermath during a memorial service in Tokyo.
"Many victims still lead difficult lives in devastated areas and places that were evacuated," he said, referring to the more than 200,000 people who are still unable to return home due to radioactive contamination.
"It is important for all people to join together and show their support in the long-term ... I pray for a return of peaceful times to devastated areas," he added.
When the undersea quake hit the northeastern coast of Japan on March 11, 2011, a wall of water pummeled the coast, sweeping away thousands of victims and destroying communities. A total of 15,884 people are confirmed to have died in the tsunami with another 2,633 still listed as missing, according to the latest government figures.
Waves also crashed into the Fukushima Dai-ichi power plant, sparking reactor meltdowns and setting off the worst nuclear crisis in a generation.
The crippled plant has also suffered several cleanup setbacks. Following the disaster, hundreds of tanks were built around the plant to store huge amounts of radioactive water leaking from the damaged reactors. Much of the work to stabilize the plant has been temporary as there is no permanent solution for this contaminated water.
Prime Minister Shinzo Abe has, however, vowed to boost the rebuilding, earmarking 25 trillion yen ($250 billion/175 billion euros) for reconstruction through March 2016.
A full decommissioning of the site is expected to take decades.
hc/mkg (AFP, AP)