Fires were set at two entrances to the building in Reichertshofen in the early hours of Thursday, resulting in damages set to cost 150,000 euros ($164,000), according to authorities. The fire mainly affected an annex of the building which had previously been used as a discotheque.
The building is scheduled to house 67 asylum seekers from September.
Police spokesman Hans-Peter Kammerer said on Thursday that "a xenophobic motive cannot be ruled out." The city council recently approved the change of use for the building when the number of asylum seekers to be housed there had been reduced.
There have been a series of protests against housing asylum seekers in the town, which is 68 kilometers (42 miles) north of Munich.
Last week a building housing 150 asylum seekers in the east German town of Boehlen in Saxony was attacked. There were shots fired at the unit but no one was injured.
Germany is currently dealing with record numbers of asylum applications.
In May, the interior ministry confirmed reports that 400,000 new applications for asylum in Germany were expected to be received before the end of the year; that is double last year's figure. Interior Minister Thomas de Maizière said the ministry could announce a higher number in July or August.
As a single country, wartorn Syria is the most common source of aslyum applications filed in Germany; behind that, Kosovo, Albania and Serbia complete the top four. Recent changes in German law dramatically reduces the chances for applicants from Balkan countries winning the right to stay.
De Maizière estimated in May that about a third of all refugees who cross the Mediterranean to Europe could arrive in Germany and submit an asylum application.
The anti-Islam group PEGIDA has organized rallies in Dresden and other cities in Germany this year.
The Alternative for Germany (AfD) party has also begun to attract voters who are against immigration, and recently underwent a change of leadership - seen as a clear step towards populist right-wing politics. Party founder Bernd Lucke - an MEP, economics professor and self-professed euroskeptic - was dethroned by his more fiery understudy Frauke Petry. It's considered possible that Lucke might establish another new party of his own.
Among EU member states, Germany remains the most popular target country for asylum seekers. In 2014, Germany processed 172,945 first-time applications, according to Eurostat, followed by Sweden (74,980), Italy (63,000), France (57,000) and the United Kingdom (31,070).
jm/msh (epd, dpa)