Investigative journalists are crucial watchdogs of society, uncovering secrets and exposing scandals. But with donors increasingly keen to fund their work, can the impact of investigative journalism be measured?
Amid suffering business models, investigative journalism promises positive repercussions for governance and society
Investigative journalism today needs to find new ways to prove its case. This may seem strange, as this special form of journalism has been extraordinarily successful in recent years. In the wake of the Panama Papers, for example, the world has seen penalties for individuals and companies in many countries as well as cases of concrete reform.
"For every dollar spent on investigative reporting, hundreds of dollars are gained for societies from changes in public policy," writes James T. Hamilton in his in-depth study into investigative journalism,Democracy’s Detectives.
But this is not enough to ensure the survival of investigative journalism. The business model with which it has traditionally been financed is collapsing. Newspapers and other legacy media have been losing their advertising revenue and thus are less and less able to pay for in-depth reporting.
Convinced of investigative journalism's positive repercussions for governance and society, different donors are willing to jump in and actively support such reporting. But in line with their funding principles, they wish to be informed about what has been achieved with the resources they made available. Meeting donor requirements is often a challenge for investigative reporters and their colleagues. How can they prove the impact of their reporting ?
What is impact?
To complicate things, the term "impact" has no consistent definition in the media context. Many media experts use it as a general term for the different kinds of effects reporting may have on audiences and societies. When it comes to investigative journalism, some experts suggest defining impact by differentiating between three different kinds of results.
In evaluation as well as in international development, "impact" is defined in a more technical, precise manner: It is understood as the highest level of change that can be achieved through an intervention.
According to theOECD, the (positive or negative) change effected by an intervention may be termed impact, if it is significant, extensive and transformative in nature. An impact can be achieved at the end of a result-chain which is based on "outputs" (products, capital goods and services), which may lead to "outcomes" (short-term and medium-term change and effects of intervention outputs) and which finally may make impact possible.
While different forms of outcome are achieved more often, impact in the stricter sense is rare in investigative journalism. And even less common is convincing evidence that lasting change is a direct result of a particular investigative report.
However, the media may be more influential overall when it comes to molding the attitude and behavior of political elites, and through its muckraking it also acts as a deterrent. Investigative reports can also lead to important discussions on the values and standards a society wants to live by, laying the foundations for more fundamental change.
Different approaches are used to assess outcomes or impacts of investigative journalism. They tackle the problem from different angles, on different levels, and with different methodologies. In the following, key approaches are briefly described.
As a first step, investigative reporters can use digital analytics tools to see how their stories are viewed, shared and commented on their websites and social media platforms. On top of this, it may then be useful to conduct agenda setting research by analyzing the way topics are presented and picked up by the media.
This then includes a study of changes in framing amongst the public with the use of modern technology. For example, MIT Center for Civic Media's "Media Cloud" assesses how a story or a topic spreads across the web and social media. The UN Pulse Lab uses artificial intelligence to analyze the content of call-in radio shows to track how public discourse evolves.
In order to manually document outcomes and impacts achieved through investigative reporting, different forms of tracking reports have been developed by non-profit organizations conducting investigative reporting. Whether this is done with a modern online tool or a simple spreadsheet is less important - what counts is that the information is gathered on different levels of outcome and impact.
ProPublica, a foundation funded, non-profit investigative media outlet, assembles an internal "tracking report", which documents information according to the following categories: "action influenced by the story", "opportunities to influence change", "change that has resulted" and "lessons learned". In some cases, this monitoring is continued for many years – as long as the organization continues to report on the particular issue.
Other examples that serve a similar function but also add various forms of tagging, categorization, and multi-use charting are the "Impact Tracker" developed by the Center of Investigative Reporting (CIR), Newslynx by the Tow Center for Digital Journalism at Columbia Journalism School and MORI (Measures of Our Reporting's Influence), an open source WordPress plug-in produced by Chalkbeat. These tools combine offers for the manual documentation of outcomes by the investigative team with some of the digital metrics described above.
After completion of extraordinary projects, narrative accounts on the successes of investigative reporting are often produced. Here, investigative reporters or their colleagues tell the story behind the story of their muckraking. ProPublica, for example, publishes an annual impact report to provide an overview of its successes in achieving outcomes and impact.
On top of this, an evaluation of change processes by independent experts can be of use. Some basic verification is performed by the jury-members of investigative reporting awards. An independent evaluation expert can go even further than simply checking the plausibility of the reporter’s accounts. Based on the documentation and interviews with different sources, an evaluator can conduct his own research and provide a new level of understanding of what was achieved and how. An example is the analysisof selected cases of African investigative journalism.
Finally, academic research offers study designs to gauge the impact of investigative journalism, such as studying coalition building between different actors before and after investigative media reports. Two prominent and convincing examples of this kind of work as well as investigative journalism in general are the books "Journalism of Outrage" by David Protess and "Democracy's Detectives" by James T. Hamilton.
Jan Lublinski is Head of DW Akademie's Policy and Learning department
Coronel, Sheila (2008): Corruption and the watchdog role of the media. In: The News Media and the Governance Agenda. World Bank/Shorenstein Center. Conference Paper. Link
Graves, Lucas / Shabbir, Nabeelah (2019): Gauging the Global Impacts of the ‘Panama Papers' Three Years Later. Reuters Institute, University of Oxford. Link
Green, Elizabeth / Cramer, Philissa / Anand, Anika (2014): What We Talk About When We Talk About Impact. Chalkbeat white paper. Link
GIJN (2014): Investigative Impact. The Role of Investigative Journalism in Fostering Change and How to Measure it. Link
Hamilton, James T. (2016): Democracy's Dectectives. The economics of investigative journalism. Link
Jurrat, Nadine (2017): Investigative Journalism. With a little help from my audience. Link
Jurrat, Nadine / Lublinski, Jan / Mong, Attila (2018): Raking Muck and Raising Funds – Capacity Development Strategies for the Future of Investigative Journalism in the Global South. In: Investigative Journalism Education Consortium. Link
Kaplan, David E. (2013): Global Investigative Journalism: Strategies for Support. A Report for the Center for International Media Assistance. 2nd edition. Link
Learning for Action (2013): Deepening Engagement for Lasting Impact. A Framework for Measuring Media Performance & Results,” published by the Gates and Knight Foundations. Link
Lublinski, Jan / Spurk, Christoph / Fleury, Jean-Marc / Labassi, Olfa / Mbarga, Gervais / Nicolas, Marie Lou / Abou Rizk, Tilda (2016): Triggering Change - How investigative journalists in Sub-Saharan Africa contribute to solving problems in society. In: Journalism. Theory, Practice and Criticism. 17(8) 1074-1094. Link
Napoli, Philip M. (2014): Measuring Media Impact. An Overview of the Field. Link
OECD/DAC (2019): Better Criteria for Better Evaluation. Revised Evaluation Criteria. Definitions and Principles for Use. OECD/DAC Network on Development Evaluation. Link
Protess, David L / Cook, Fay Lomax / Doppelt, Jack C. et al. (1991): The Journalism of Outrage. Investigative Reporting and Agenda Building in America.
Schiffrin, Anya (ed., 2014). Global Muckraking. 100 Years of investigative journalism around the world.
Schiffrin Anya (2014). Measuring Media Impact. Taking the long term view, supporting media independence. Link
Schiffrin, Anya / Lugalambi, George William (eds., 2017). African Muckraking.
Schiffrin, Anya / Zuckerman, Ethan (2015): Can We Measure Media Impact? Surveying the Field. In: Standford Social innovation Review. Link
Stetka, Vaclav / Örnebring, Henrik (2013): Investigative Journalism in Central and Eastern Europe: Autonomy, Business Models, and Democratic Roles. In: The International Journal of Press/Politics 18 (4) 4113-435.
Tofel, Richard J. (2013) Non-profit journalism. Issues around impact. A whitepaper from ProPublica. Link
UN (2015): UN Global Pulse Lab. Analysing radio content. Link
Young, Nany / Suarez, Cyndi (2017): Unintended consequences. Should We Measure the Impact of Investigative Journalism. In: Columbia Journalism Review July 19. Link
Zhong, Nicole Zhaoqui (2018): Making impact a Key Performance Indicator. Link