The dispute has been going on since the 1990's when some of Ukraine's Orthodox Christians broke away from the Russian-controlled church in Moscow and formed their own autonomous church.
Ukrainian President Viktor Yushchenko supports the idea of a separate church, but the head the Russian Orthodox Church, Patriarch Kirill, is unwilling to consider such a move.
The two men clashed on Monday as Yushchenko repeated his call for a Ukrainian Orthodox Church independent of Moscow.
"Of course the greatest hope of the Ukrainian people is to live under a unified and independent church," said the Ukrainian leader.
"This church already exists, and if it did not, Ukraine would not exist," Kirill hit back firmly.
The Patriarch started his visit in Ukraine by leading a prayer service at a monument to Saint Vladimir, the prince of Kiev, whose conversion in the tenth century was the start of Orthodox Christianity in Ukraine and Russia.
"If we all pray together to overcome divisions, sorrow and confrontation we will have a peaceful, calm and joyous life," said Kirill at the service.
The service was attended by about 2,000 worshippers. At the same time dozens of Ukrainian nationalists held a protest nearby and shouted, "Go away Moscow pope!"
The theological dispute comes on top of other issues that Moscow and Kiev have been having, such as the transit of natural gas from Russia to Europe or Ukraine's desire to join NATO.
While the country's main Orthodox church still answers to Kirill, several church leaders have already proclaimed their independence from Moscow and have been gaining popularity as well as support from the Ukrainian president.
The Russian Orthodox Church claims about 28 million followers and the separatist Ukrainian Orthodox Church has about 14 million believers.
This is Kirill's first visit to Ukraine since he became the head of the Russian Orthodox Church in February.
Editor: Neil King