Russian President Vladimir Putin and Japanese Prime Minister Shinzo Abe agreed last November to step up talks on the peace treaty based on a key 1956 joint declaration.
The declaration requires that Moscow hand over two of the four disputed islands -- Shikotan and the Habomai islet group -- once a peace treaty is concluded, Presstv Reported.
However, Russia is concerned that President Vladimir Putin's support rate may drop if it cedes the islands off Japan's northernmost main island of Hokkaido, Kyodo cited what it called diplomatic sources as saying.
According to the news agency, Moscow has been reluctant to return the islands, saying the presence of US military forces in Japan remains a threat.
Opposition lawmakers in Japan have also condemned the agreement and attacked the Japanese premier for seeking the return of two of the islands rather than all four.
The two countries have sought to resolve the prolonged territorial dispute as it remains a sticking point in normalizing ties, more than 70 years after the end of World War II.
However, little progress has been made since the November agreement. Kyodo said Russia's reluctance to start negotiations even on the return of the two islands indicates the negotiations will go nowhere at least for the time being.
The Japanese government once aimed to reach a broad peace treaty by the time of the meeting between Abe and Putin on the sidelines of the Group of 20 summit in Osaka, but it gave up on the attempt due to persisting differences over how to settle the territorial dispute.
During the summit last month, Abe and Putin made little headway in their talks and agreed only to continue negotiations on the territorial dispute.
The Soviet Union seized the islands straddling the Sea of Okhotsk and the Pacific Ocean after Japan's surrender in the war in 1945.
Earlier this month, the Japanese ministry of foreign affairs described the islands as "an inherent part of the territory of Japan, which have never been held by foreign countries.”
Moscow has constantly stated that Russia’s sovereignty over the islands cannot be called into question. Opinion polls also show most Russians oppose handing back the islands to Japan.