The far-right AfD has been Chancellor Angela Merkel's fiercest critic and repeatedly called for her removal from office. As she kicked off her fourth term, Merkel made it clear that she is eyeing their removal as well.
She emphasized, however, that her government's main goal is to solve the problems plaguing German voters.
Merkel was re-elected for a fourth term as chancellor by a margin of just nine votes.
Although the ballots are secret, the results indicated that lawmakers within the governing grand coalition between her conservative Christian bloc and the center-left Social Democrats (SPD) voted against her.
Merkel's Christian Democratic Union (CDU) and the SPD took a hit during last September's general election, losing voters to the AfD. The far-right, populist party entered the Bundestag for the first time after earning 12.6 percent of the vote. They are currently the largest opposition party.
AfD politicians particularly disapprove of Merkel's refugee policies and have repeatedly called for Merkel's ousting from office.
During the vote in parliament on Wednesday, an assistant to an AfD deputy was expelled from the spectator's balcony after unfurling a banner that read "Merkel must go."
Another AfD lawmaker was fined for posting a picture of his ballot on Twitter, showing that he'd voted against Merkel, along with the message: "Not my chancellor." Parliamentary rules dictate that the votes must remain secret.