Murwanashyaka was sentenced to 13 years behind bars for aiding and abetting war crimes, while Musron was given an eight year jail term, in a landmark case by a court in Stuttgart.
The pair who led the Democratic Forces for the Liberation of Rwanda (FDLR) were arrested in Germany in 2009 and went on trial in 2011.
The court heard how their ethnic Hutu rebel group, which was formed in the 1990s, raped and massacred hundreds of people in the Democratic Republic of Congo as their fighters resisted an assault by the Congolese and Rwandan army in 2009.
Hutu militiamen terrorized civilians in the Kivu region of Congo while exploiting the area's precious minerals. Several villages were attacked and many people were killed when their huts were set on fire.
Despite living in asylum in Germany for more than 20 years, prosecutors said Murwanashyka and Musoni remained at the helm of the FDLR's activities from a distance.
The trial initially centered around 26 counts of crimes against humanity and 39 counts of war crimes committed by militias under their command between January 2008 and their arrest in November 2009.
The pair's defense argued that as a politician, he had no influence on the military arm of the FDLR that committed the atrocities.
Victims of the FDLR militia were heard via video telephone while others were flown from Rwanda to Germany.
The trial was the first under the Code of Crimes Against International Law, adopted in 2002, which allows German courts to investigate and prosecute severe cases wherever they are committed in the world.
Prosecutors had pushed for a life sentence for Murwanashyaka with no conditional release after 15 years and 12 years in prison for Musoni.
Rwanda has previously said it would seek Murwanashyka's extradition for alleged crimes committed during the early 1990s Rwandan Genocide.
mm/rg (DW, dpa)