Why fact-checking is the root of journalism: 8 good questions with PolitiFact’s Angie Drobnic Holan

Angie Drobnic Holan is top editor in charge of PolitiFact, a news unit within the Tampa Bay Times organization. She sees fact-checking as a part of journalists’ mission to hold politicians accountable.

While technology has changed quickly and challenged the work of journalists, the roots of journalism have not changed and may even become more important. We talked with Holan about reader trust and how fact-checking tools like the Truth-O-Meter can help reach out to a new audience.

1. Take us back in time a little bit — how did the idea for the Truth-O-Meter come about? 

Angie Drobnic Holan

HOLAN: We started PolitiFact and our fact-checking Truth-O-Meter back in 2007. That was an interesting time for us as a news organization because our Washington bureau chief, Bill Adair, was working for the Tampa Bay Times and thinking about how to cover the 2008 presidential election in a new and different way.

He knew he wanted to do something new with more fact-checking because he had a personal belief that there wasn’t enough fact-checking in political news, and one of the ideas he had was for a database-driven website.

During 2007 when we were planning our election coverage, he pitched this idea to Tampa Bay Times editor Neil Brown and asked if he could launch this site, and the idea was approved. And it’s been very successful ever since.

The way it works is that we take individual statements from politicians, candidates, and now even sometimes talking heads, and we fact-check the statement and then we give it a rating to share its relative accuracy. And our ratings are true, mostly true, half true, mostly false, false, and then for a statement that is false and ridiculous we give it the rating pants on fire.

2. How do you decide which statements to fact-check?

We select statements to check primarily based on whether we think something either sounds wrong or just catches our attention and makes us wonder, “Hmm, could that be true?” We try to pick the things that we think the average voter would be interested in, or care about. And then, after those criteria, we follow the news, we try to fact-check whatever is being talked about in the national discourse.

So when we come in every morning, we have discussion about what’s been said, about what the issues of the day are. We have an internal spreadsheet where we collect facts that sound like we might like to check them and then we just basically look them over and say, what seems like it would be the best news story for a fact-check? And then we go forward.

We try to fact-check approximately the same number of Democrats and Republicans but we don’t keep hard-and-fast count, and one thing that we don’t do is try to balance the ratings. We don’t think about if we get a false on one side, we want to go and get a false on the other side. We do not do that. We just try to pick an interesting, diverse mix of facts to check. And then wherever the ratings come out in our editorial process, that’s how they come out.

3. Are there any red flags or especially recurring things you look for?

If we hear something that sounds wrong that we say, “Hmm, that sounds wrong,” we will definitely be attracted to fact-checking that. Now sometimes something will sound wrong reported, and we search it, and find that it’s actually not wrong, and we’ll still publish it anyway. We are a little different than some fact-checkers in that because we have the true rating and the mostly true rating, we can still publish our findings if something turns out to be true and sometimes it does.

In other cases it’s just facts that sound interesting or intrigued to us at some level and we’ll fact-check those to find out because you know, we are journalists and I think like a lot of journalists we are driven by curiosity and willing to find out more about the world. So some of our fact-checks are just some things that we hear and found interesting and we wanted see what the story behind them was.

4. You are working with partners across the country, how does everybody draw consistent conclusions? Can this be entirely objective?

We have a process for reporting, and the way it works is that one reporter is on a story and they work with their editor and the editor and the reporter then bring the story to a conclusion. For that process that’s very much like a traditionally reported news story; we review documents, we interview experts.

What’s different between PolitiFact and traditional news stories is, in a PolitiFact news story we go through all of the evidence we find in a pretty detailed fashion and we also in our online reports include all of our sources. So there’s a list with links to all the documents we reviewed and we name all of the experts we spoke with.

At the end of the process we have a panel of three editors who vote on the rating and sometimes there is disagreement about whether something should be rated say half true or mostly true or whether something is false or mostly false. And the editors-panel votes on this finding.

Now we’ve published more than 7,000 fact-checks and we don’t expect readers to agree with every rating. Sometimes they disagree strongly, but overall we feel like we are very transparent in our reporting and our methods, so even if someone disagrees with our rating, we hope they’ve learned some things from our report.

5. Have you seen politicians stop making false statements as a result of your fact-checking?

You know, it’s interesting because I do see some politicians have changed their talking point after we fact-check it or others have fact-checked it. Other politicians just seem to repeat the same thing regardless how often it’s fact-checked, so I think it depends on the specific case.

However, I would say most of the people we fact-check want to be right and when they are found that their statements are wrong sometimes they’ll say that maybe they misused the words but their broader point was accurate. So in these cases sometimes we will see them change what they say to try to make it more accurate.

Overall I do think fact-checking has been very helpful to journalism because I think it has brought renewed attention to the importance of checking facts and not repeating claims that are easy to verify if they are not accurate, though maybe “easy” isn’t the right word. I think journalists want to be careful to not repeat claims that are not accurate. I think fact-checking gives a renewed emphasis to that and I see in traditional news stories more fact-checking since I started fact-checking back in 2007, and that is very heartening.

6. How do you measure the impact of your work?

You know, impact is an interesting question. I say often that we do not fact-check with the intention to stop politicians from exaggerating because I think that might be just part of politicians and what they do. And in fairness to the politicians, I think human beings, all of us, maybe had situations sometimes where we tended to exaggerate.

What we say is, we are fact-checking to give voters the information they need to govern themselves. So we are not really trying to change anyone’s behavior but we are fact-checking hoping that voters get the information they need so that when it’s time to make decisions about who to vote for, or where they stand on a particular issue, that they have good, factual, and accurate information to make those decisions.

7. Do you help teach other journalists about how to fact-check?

We train people who are our formal partners, who are part of our PolitiFact network. With international journalists, we tend to be more informal advisors, or have discussions with them. Another thing is, you know, we have a small staff, so our first priority is to publish our own fact-checks. So our time is limited as far as training goes but we do try to spread the word about fact-checking, share our techniques. We are always happy to talk about how we do things.

We’ve also published our reports online about how we do our fact-checking, about what our methods are, we have a document on our website called “Principles of the Truth-O-Meter” that explains in some detail how we approach fact-checking.

8. Looking ahead, do you see potential in the Truth-O-Meter expanding, either by franchising it or having teams abroad?

We have 10 state affiliates right now and we are always looking for new affiliates. We would like to have affiliates in all 50 states to fact-check the state-based elections, so I do think that there is lots of opportunity for expansion there.

We also have one international partner. We have helped an organization, PolitiFact Australia, to get launched in Australia, and they fact-checked the most recent election and we are very pleased with the results.

In addition to that, we have also advised several organizations about how to launch international fact-checking efforts, but language barriers made it more difficult to do formal partnerships. But we certainly see fact-check efforts growing around the world, and we support that because it’s independent journalism that has a focus on accuracy and and accountability. So I think it has a lot of potential to expand. I think we are really just starting right now.

Journalists we’ve talked to in other countries about what we’re doing sometimes just start sites on their own and they will say that they were influenced by PolitiFact. I would emphasise that fact-checking has its roots in traditional journalism. There is nothing that is secret about how to fact-check. I think it’s more just of an approach, an outlook, and also a commitment from the news organization that it wants to fact-check.