The first wave of the tsunami triggered by the powerful earthquake in Chile hit Japan's outlying islands at around 12:48 a.m. local time, (0348 GMT) Sunday, but the initial waves were just 10 cm high.
The waves first hit a small island of Ogasawara islands 1,950 km south of Tokyo, public broadcaster NHK reported. The Japan Meteorological Agency (JMA) predicted it will soon reach other parts of the Pacific coastline of the country.
Waves of up to 30 cm were seen in the eastern coast of Hokkaido at around 2 p.m.. There had been no reports of damage so far.
Local governments had urged thousands of households in northeast Japan to evacuate, where the waves are expected to be more than 3 meters high.
Japanese Prime Minister Yukio Hatoyama appealed to people not to go near to the coastal areas and stay in the shelters until the evacuation orders are lifted.
"We must not drop our guard and please don't go near the coast, " Hatoyama said on TV.
Major tsunami alert is issued by the JMA for the pacific coast of Aomori prefecture, coast of Miyagi and Iwate prefectures in northeast Japan, saying "very high tsunami" with waves up to 3 meters are expected in the areas, and "everyone near the coast must immediately evacuate to higher ground".
It is the first time in 17 years for Japan to issue a major tsunami alert, a warning with the most intense level.
Tsunami warnings were also issued for other areas along Japan's Pacific coastline from Hokkaido to Okinawa.
Transportation on many lines of the country's railways and highways has also been suspended due to the tsunami.
An official of the weather agency's earthquake and tsunami observation section said in the NHK that the agency may have to keep the warnings in place for a long time because "the second and third waves could be bigger than the first waves."
In May 1960, a tsunami struck the coasts of Hokkaido and the Sanriku region of Japan after the huge M9.5 quake in Chile, leaving around 140 people dead or missing.
An earthquake with a preliminary magnitude of 8.8 struck Chile Saturday, killing so far at least 300 people.