Four segments of the wall, which is part of a 1.3-kilometer (nearly 1 mile) stretch known as the East Side Gallery, were removed Wednesday morning at around 5:00 a.m. local time (0400 UTC). The work crews were backed by around 250 police.
The 1.2-meter-wide (4-foot) wall segments were taken out to make way for an access route to planned high-rise luxury apartments.
The projects investor, Maik Uwe Hinkel, made the decision to remove the wall segments, police spokesman Alexander Tönnies said.
"The developer had the right to do this and he informed us a few days ago about his plans. Last night we were told that he wanted to remove the wall pieces early this morning," Tönnies said, adding the Wednesday's removal took place without incident.
Protesters who want the wall to remain have gathered in the thousands since the beginning of March, when the first panel was taken away. The wall's removal was halted shortly afterwards.
Even US actor and singer David Hasselhoff, who gave a legendary New Year's Eve performance of his song "Looking for Freedom" at the Wall after its fall in 1989, joined the demonstrations.
The East Side Gallery is the Berlin Wall's longest surviving stretch. Since 1990, the outdoor art exhibit has been covered in brightly colored graffiti murals painted by artists from around the world. One famous mural includes the "Fraternal Kiss" depicting the famous embrace between Soviet leader Leonid Brezhnev and East Germany's Erich Hoenicker. The gallery has become a major attraction for tourists visiting the German capital.
"I can't believe they came here in the dark in such a sneaky manner," Kai Alavi, the head of the East Side Gallery's Artists' group, told the AP news agency. "All they see is their money, they have no understanding for the historic relevance and art of this place."
Developers have said that plans to provide access to a 63-meter-high residential development along the Spree River, as well as access to a planned bridge, had required a 22-meter portion of the Wall to be removed.
A meeting between the property company and city authorities on Tuesday about possible alternative sites for the development proved fruitless.
The Berlin Wall was first erected in 1961. Dividing the city's East and West German portions, it spanned 155 kilometers (96 miles) in total. At least 136 people died trying to scale the Wall until it was torn down 28 years later on November 9, 1989.
dr/hc (AFP, dpa)