He said the world is increasingly becoming multipolar and that Europe should look for more global cooperation on equal terms.
"Those who are nostalgic for the dream of a European world power, those who serve national superpower fantasies, are stuck in the past," Scholz said in his speech on Tuesday.
This is in contrast to French President Emmanuel Macron who recently called for the EU to become a global power.
Scholz, however, echoed some of the themes Macron raised, saying that "the United States remains Europe's most important ally." That relationship could be strengthened if the EU invested more in security and defense, civil resilience, technological sovereignty and independent sources of crucial commodities.
EU must not be 'intimidated' by Putin
The speech comes on Europe Day, when the continent celebrates the years of peace in the bloc that followed the end of World War II fighting in Europe.
Europe will only be heard if it is speaking with one voice, Scholz said.
"Russia's brutal war of aggression against Ukraine has shown us how essential this realisation is," he added, saying the EU should aim for much closer military cooperation.
He delivered his speech in Strasbourg on the same day that Russia was celebrating Victory Day.
"In Moscow, 2,200 kilometres northeast from here, Putin is parading his soldiers, tanks and rockets. Let us not be intimidated by such a show of force," Scholz told the European Parliament, referring to Russian President Vladimir Putin.
"Let's stay steadfast in our support for Ukraine — as long as it is necessary," he said.
Efforts to rebuild Ukraine will need "political and financial capital over the long term," he said.
Scholz on ties to China
The invasion has highlighted the EU's, and particularly Germany's, dependence on energy imports from Moscow. There are similar fears about an overreliance on China for manufacturing
The German leader sees China as a partner despite shifts in relations.
"Our relationship with China is aptly described by the triad 'partner, competitor, systemic rival' — where rivalry and competition on China's part has undoubtedly increased," he said.
Despite this Scholz said he agreed with European Commission President Ursula von der Leyen's idea that Europe should reduce risks linked to its economic dependencies from Beijing.
Scholz said there was also a need for reform within the EU.
Notably, he wants decisions including on foreign relations and takes to be taken by the majority rather than unanimity.
Free trade and the EU's promise to Balkans and Africa
Scholz also touched on topics that included the freed trade, accession of new members and migration.
"It makes a lot of sense to swiftly seal new free trade agreements with Mercosur, Mexico, India, Indonesia, Australia, Kenya and - over the long term - with many other countries," Scholz said.
"If we continue fruitless negotiations over new free trade deals for years to come, others will dictate the rules - with lower environmental and social standards," he warned.
He said the EU should keep its promise toward prospective members such as the Western Balkans countries, while ensuring they undertake the necessary reforms to be able to join the bloc.
Addressing the EU's relations with Africa the chancellor said partnerships should "leave behind the Eurocentric view of past
"Overcoming the consequences of colonialism must be an essential feature of any European partnership with the countries of Africa,
Asia and Latin America," Scholz said.
He also said the EU need to address long-running disputes over migration stressing that many parts of Europe urgently needed labor from outside the bloc.
Reform of Europe's asylum system should happen before next year's European Parliament elections according to Scholz.
Some advance criticism
In advance of the speech, Scholz's critics within the European Parliament noted some of Germany's actions had not been popular in the legislature.
Manfred Weber, from the center-right European People's Party group, accused Scholz's politically broad "traffic light" coalition of Social Democrats, Greens, and neoliberal Free Democrats of failing to have a "clear and agreed position" on important issues.
Weber criticized Berlin's policies on China and said Scholz's speech was a chance to "repair some of the damage done."
The European lawmaker cited Germany's last-minute questioning of an agreement to phase out vehicles with internal combustion engines as damaging Germany's credibility.
Greens European Parliament group leader Terry Reintke also cited this as an issue that had eroded trust in Berlin and added that many of her colleagues from eastern and central Europe had been disappointed with the speed of Scholz's response to events in Ukraine.
Reintke said the perception was "that everything came very slowly and that it had to be wrested from Berlin again and again."
The address was set to be followed by a debate with members of the parliament.
lo,rc/sms (dpa, Reuters, AFP, AP)