"The Somali forces and AMISOM peacekeepers secured control of Marka again and now the situation has returned to normal," Abdirisak Mohamed, a Somali military official, told the AFP news agency by telephone.
"There was a brief exchange of gunfire, but the militants have fled," he said, adding that "several" al-Shabab fighters as well as one Somali soldier were killed during the clashes.
"The AU and Somali troops attacked the town from two directions, forcing the terrorists ... out of town," Mohamed Hussein Shine, a local official, reported.
A Marka resident who spoke with AFP said the AU troops - also known under the acronym AMISOM - used tanks to recapture the port, adding that four civilians were killed after they were caught in the fighting.
"Four people died in our neighborhood and two others were wounded. I can see AMISOM and Somali troops returned to the city now, and they are conducting security operations," Muhidin Osman said.
Further witnesses stated that many people fled their homes due to heavy shelling and that the al-Shabab militants set a police station on fire before leaving the city.
Sharia law imposed
After seizing the city on Friday, the al Qaeda-linked al-Shabab group said that one of its leaders, Sheikh Mohamed Abu Abdallah, had addressed hundreds of people gathered at the regional government headquarters in Marka.
"The enemy has lost and (is) running away. They are fleeing from the Islamic regions," they quoted him as saying.
Prior to the AU and Somali troop intervention in the city, witnesses reported that the militants imposed Sharia law on the people of Marka.
"Shabab fighters are patrolling in the streets, and they asked people to avoid engaging in anything that could be a breach of the Sharia," Marka resident Mohamed Mowlid said earlier on Saturday.
The historic port of Marka, around 100 kilometers (60 miles) south of the capital, Mogadishu, was captured in August 2012 by AU troops after four years under Islamist control. The al-Shabab militant group has been fighting to overthrow Somalia's internationally backed government, which is protected by 22,000 AU troops.
Friday's loss of Marka was seen as a heavy blow to Somali government and AU troops, which have suffered several major attacks from al-Shabab in recent months, including shootings, suicide bombs, and raids on AU bases.