The Italian capital was bidding to host the 2024 Games but Virginia Raggi says continuing would be irresponsible in light of debt concerns. It is a further blow to the IOC after Boston and Hamburg also shelved bids.
The new Rome mayor has gone against Italian Prime Minister Matteo Renzi and decided not to back the bid.
"It would be irresponsible for us to support this candidacy," said Raggi, who took over in June.
"We have nothing against the Olympics and sport ... but we don't want sport to be an excuse for more rivers of cement in the city. We won't allow that. We won't be forced to pay for more cathedrals in the desert for years to come. Romans don't want that," added Raggi.
The 38-year-old also tweeted the same statement.
Although not official, it surely means the bid is dead.
That would leave only Paris, Los Angeles and Budapest as candidates with a decision due in September 2017.
The International Olympic Committee is concerned that its flagship product is losing its worldwide appeal after a raft of cities ended Summer and Winter Games bids in recent years.
IOC president Thomas Bach launched an initiative to reduce the costs of the sporting extravaganza but it is yet to bear fruit.
The Rio Olympics in August, which cost around $12 billion, was deemed a success but was dogged by controversy and doubts remain about its legacy.
Budget concerns also led Rome to pull out of the running for the 2020 Games, which will be held in Tokyo.
mm/jh (rtrs, dpa)