Major General Godefroid Niyombare said on Tuesday that the president had violated the national constitution and the Arusha peace accords by seeking a third term in June elections.
"Given the necessity to preserve the country's integrity ... the president, Pierre Nkunrunzia, is dismissed from his functions," Niyombare said in a statement on Bonesha FM radio.
Niyombare told reporters that he was working with civil society groups, religious leaders and politicians to form a transitional government after dismissing the president. The general said he himself would preside over a committee to re-establish stability.
People streamed, cheering and singing, onto the streets of the Burundian capital after the announcement. The declaration was made as Nkurunziza was in Tanzania to attend an East African Community summit on the crisis in his country.
Rubbished as 'a joke'
The veracity of his comments and the extent to which he was backed by the military was called into question.
The presidential office swiftly dismissed as "a joke" the statement by Niyombare, who was fired by Nkurunziza as intelligence chief in February.
A South African Foreign Ministry spokesman said it was "way too early" to determine whether a coup had actually taken place.
Niyombare, who is also a former ambassador to Kenya, was surrounded by several other senior army and police officials, including a former defense minister, when he made the statement.
rc/mkg ( Reuters, dpa, AFP, AP)