The amount of fact-checking journalism produced in the United States is increasing dramatically, and while there are limits to its persuasiveness, it is a measurably effective tool for correcting political misinformation among voters, according to new scholarly research conducted for the American Press Institute and released today.
The number of fact-check stories in the U.S. news media increased by more than 300 percent from 2008 to 2012, one of the studies found. That accelerates the growth in fact-checking journalism found in the prior national election cycle.
Fact-checking journalism also succeeds in increasing voter knowledge, according to controlled experiments with audiences.
“Fact-checking journalism is growing rapidly but is still relatively rare and heavily concentrated among outlets with dedicated fact checkers,” said the University of Exeter’s Jason Reifler, one of the scholars engaged in the research.
[pulldata align=center stat=”300%” context=”increase in fact-check stories”]The three studies released today, conducted by scholars at six universities, build on existing research and constitute the most comprehensive effort to date examining the work of journalists to police political rhetoric.
Among some of the other findings:
- More than eight in 10 Americans have a favorable view of political fact-checking.
- Fact-checking is equally persuasive whether or not it uses a “rating scale” to summarize its findings.
- Fact-checks of inaccurate statements are more persuasive when the consumer and the politician belong to the same political party.
- Democrats, in general, have a more favorable view of and are somewhat more persuaded by fact-checking journalism than Republicans.
The results released today are part of a series commissioned through API’s Fact-Checking Project, an initiative to examine and improve fact-checking in journalism. The program is funded by the Democracy Fund, the William and Flora Hewlett Foundation and the Rita Allen Foundation.
The growth of fact-checking
By several measures fact-checking is growing. In the study of the frequency of fact-checking — either original fact-checks or stories about such work — the number of fact-checking stories increased by more than 50 percent from 2004 to 2008 and by more than 300 percent from 2008 to 2012. The growth occurred mostly at 11 newspapers that partnered with PolitiFact, one of the country’s most prominent fact-checking organizations, but the number of such stories also more than doubled between 2008 and 2012 at media outlets unaffiliated with PolitiFact.
The findings on the growth in fact-checking are reinforced by the Reporters’ Lab at Duke University, which found that the number of fully active fact-checking organizations in North America increased from 15 in April 2014 to 22 in January 2015.
The API study, authored by Lucas Graves at the University of Wisconsin, Brendan Nyhan at Dartmouth College and Reifler, also explored what conditions encourage more fact-checking journalism to occur. The researchers found that reporters who are reminded of fact-checking’s journalistic value produce significantly more fact-checking stories than those who are not reminded. Yet, the study found, reminding reporters that readers like fact-checking did not have a statistically significant effect.
Fact-checking and consumer knowledge
A second study, also by Nyhan and Reifler, found that more than eight in 10 Americans have a favorable view of political fact-checking journalism.
But there are some partisan differences in public perceptions of the practice: Republicans don’t view fact-checking journalism as favorably as Democrats do, especially among people with high levels of political knowledge.
Americans also appear to learn from fact-checks written by journalists, the study found. Knowledge of relevant facts increased by 11 percentage points among people who were randomly exposed to a series of fact-checks during the 2014 election, compared to a control group. In general, the study found, fact-checks are more effective among people who already have higher levels of political knowledge.
“…political fact checking may be of particular benefit during primary contests.”
The study is the first randomized controlled trial estimating the effects of exposure to fact checking over time.
‘Pants on Fire’ optional
Another of the studies examined the effectiveness of “rating scales” in fact-checking journalism. This research, conducted by Michelle A. Amazeen of Rider University, with Graves, Emily Thorson of George Washington University, and Ashley Muddiman of the University of Wyoming, found that a fact check is an effective tool for correcting political misinformation, whether or not it employs a “rating scale.” When given a choice, however, readers selected a fact check with a rating scale.
Such ratings are used by fact-checking organizations such the Washington Post’s Fact Checker, which uses a Pinocchio scale, and PolitiFact, whose Truth-O-Meter includes the well-known “Pants on Fire” rating.
Fact-checks of inaccurate statements are less persuasive when the reader and politician belongs to opposite political parties, the researchers found. These readers tend to think the opposing party politician’s statement was false, even before they read the correction. For this reason, political fact-checking may be of particular benefit during primary contests, according to the authors, although fact-checking currently is more likely to occur during general election cycles than in primaries.
| No fact check | Text-only fact check | Text and rating fact check | |
|---|---|---|---|
| Non-political speaker | 39.21 | 32.78 | 30.88 |
| Politician | 42.08 | 39.73 | 40.25 |
Data Source: "A Comparison of Correction Formats: The Effectiveness and Effects of Rating Scale versus Contextual Corrections on Misinformation." Michelle A. Amazeen, Emily Thorson, Ashley Muddiman, Lucas Graves. February 2015
American Press Institute
The study also found that a non-political correction — in this case, regarding a statement made by a breakfast cereal company official — was more effective when a rating scale was added to the text.
The future of fact-checking
Overall the studies suggest that fact-checking is achieving its core aim: countering the spread of political misinformation. And the public largely appreciates this work.
“The results suggest that corrections of misinformation do help people to more accurately understand the world around them,” Amazeen said.
Reifler added, “In short, people like fact-checking and it appears to help them become better informed.”
Read the full studies here:
Estimating-Fact-Checkings-Effects (updated August 2016)
The Effectiveness of Rating Scales
In the coming weeks, API will publish more findings from its fact-checking research, including the prevalence of misinformation on Twitter and a report by journalist Mark Stencel examining the impact of fact-checking on the behavior of those in the political arena.
For questions about the American Press Institute’s Fact-Checking Project or for more information about the research reports, contact Jane Elizabeth, API senior research manager, jane.elizabeth@pressinstitute.org.
You might also be interested in:
The press will be much more effective in serving people and strengthening democracy if it learns from what researchers are learning. Among the examples and takeaways, you will find that news leaders and non-news experts alike value the opportunity to think differently about the challenges in front of them, about how local news can change and how research can ask different questions.
We'll share some of the resources, tools and lessons learned from our training sessions and research help desk. We hope you can use these as you plan your continuing accountability coverage and start thinking about the next election on the horizon.
When community members are no longer voters, their needs become diffuse once again and there is no clear, focusing mandate. So many newsrooms slip back into the usual: politics coverage driven by politicians and press releases. How do we avoid that backslide?
How can we avoid that backslide this time?
