"The aim of these measures is to limit the current inflows to Germany and to return to orderly procedures when people enter the country," German Interior Minister Thomas de Maiziere said. "This is also urgently necessary for security reasons."
Berlin has become increasingly frustrated with the reluctance of many within the 28-nation European Union - especially those in the former Eastern bloc - to share the burden of hosting refugees.
Germany is a preferred destination for many people fleeing Syria's civil war and other troubled nations in the migration crisis in which despite entreaties by the European Commission remains divided on a common response.
The interior ministry has not specified how long the border controls would remain in place or how exactly incoming migrants would be handled, saying only that Germany will continue to observe national and European rules on protecting refugees.
Germany's national railway, Deutsche Bahn, said it had suspended service from Austria to Germany for 12 hours at the orders of authorities as most new arrivals have arrived by train.
About 450,000 migrants have entered Germany so far this year and a total of at least 800,000 are expected this year - by far the most in the EU.
'Hungary a place of humiliation'
Meanwhile in Hungary, refugees continue to stream in from Serbia in an attempt to make it into the EU before the border fence is completed and Hungarian soldiers seal the border on Tuesday.
For only a 30 meter (98 feet) gap remains in Hungary's new barbed-wire barrier, ready to be completed at a moment's notice.
Refugees who are barred from crossing face a mammoth detour around Hungary to Croatia or Romania. And there are many on the way: officials on the Greece-Macedonia border say 5,000 people crossed overnight on Saturday.
"Hungary has become a place of humiliation and suffering for these asylum seekers," Peter Bouckaert of Human Rights Watch told AFP. "They have come from war zones. They're not just going to turn around because there's a fence, and we can't build fences all around Europe."
Berlin's move was also welcomed by Hungary's hardline Prime Minister Viktor Orban, who has made statements against refugees - especially Muslims - being a threat to Europe.
"We understand that this decision was necessary in order to defend Germany's and Europe's values," he told Germany's Bild newspaper.
Europe a continent divided over crisis
Germany's decision has been a blow to the passport-free travel enshrined in the Schengen Treaty and a landmark achievement in European integration.
Under the treaty passport controls can be re-introduced, provided they are only temporary.
"The free movement of people under Schengen is a unique symbol of European integration," the EU's executive Commission said in a statement. "However, the other side of the coin is a better joint management of our external borders and more solidarity in coping with the refugee crisis."
At an emergency meeting scheduled for Monday, interior ministers from the EU's 28 member states will discuss Commission proposals to redistribute about 160,000 asylum seekers across the bloc.
"We need swift progress on the Commission's proposals now," the Commission said in a statement issued as tens of thousands of people fleeing war and poverty in Syria and other parts of the Middle East, Asia and Africa made their way north.
EU envoys meeting on Sunday evening in Brussels failed to break the deadlock, with some eastern states still refusing to accept binding quotas of refugees.
Amid the bickering among European nations, the crisis claimed yet more lives. Some 34 refugees, almost half of them babies and children, drowned off a Greek island when their boat sank, the Greek coastguard said Sunday.
The latest sea tragedy has not daunted refugees desperate to cross. In the space of 90 minutes, a news photographer saw 10 dinghies packed with refugees arriving from Turkey on the Greek island of Lesbos, Reuters news agency reported.
Further up the refugee route, 8,500 migrants entered Macedonia from Greece between Saturday evening and Sunday afternoon, the UNHCR said.