Fact-checking “S-Town”: 5 good questions with Benjamin Phelan

“S-Town,” the latest podcast from “Serial” and “This American Life,” is the entrancing story of the brilliant-but-tortured John B McLemore and his complicated life in Woodstock, Alabama. In its first month, the truly addictive podcast was downloaded more than 40 million times. And if you’ve listened to all seven chapters, it might have seemed as if there were at least that many facts to check.

The mind-boggling job of fact-checker went to Ben Phelan, a researcher for “This American Life.” Ben agreed to relive those grueling months and answer a few fact-checking questions here for the American Press Institute.

Fact-checking a serial podcast, especially as it began breaking records for  numbers of downloads, must have been a nerve-wracking experience. What in your background prepared you for this type of intense fact-checking?

Fact-checking is weird and I suppose it can be nerve-wracking, but above all else it’s tiring. You have to maintain unnatural postures of mind for a long time, suspending all these deep and natural cognitive reflexes, such as making assumptions. That’s not too tough when you’re working on a story that you don’t like; you feel you can improve it, or at least goad the writer a little, because unlike good writers, cruddy writers resist you and fight you and want you to go away. When the writer is someone you admire, or the story gets its hooks in you, there is I think a natural impulse to back away and leave the thing alone. You don’t want to get your mitts all over it and mess it up.

But you are there to interrogate the living hell out of every single utterance of fact, and a good writer wants you to do it, and is relying on you to do it. It can feel antagonistic and gross. Well, too bad. The alternative is to give your blessing to untruth, which feels intensely grosser.

You also have to adopt and embody two antithetical traits simultaneously. You have to become an excruciatingly stupid robot, for whom nothing is obvious until it is explained and supported with minutely marshaled data points, and the wise philosopher who recognizes that truth is, ultimately, socially constructed, and who will tolerate hours-long inquiries into how we know what we think we know. A wise man knows that he does not know.

But you are there to interrogate the living hell out of every single utterance of fact, and a good writer wants you to do it, and is relying on you to do it.

The basic techniques of journalism are a great machine for extracting truth from this abyss: What do you know? What are your sources, and are they in a position to know? Find two independent sources. These practices don’t confer metaphysical certainty, but they do constitute a good operational definition of truth, and anyway they are the professional standard, so as long as you follow them, no one can fault you.

You lean into your paranoia and let it engulf you, then do the things a professional is supposed to do and find two sources. You write them on the page. Then you walk away. Next fact.

What prepared me for checking S-town was the realization, born of having worked on a whole bunch of other stories in the past, that fact-checking a treatise on, say, the black hole information-loss problem (is it really a problem, though?) is fundamentally no different from fact-checking 13 Things Jennifer Lawrence Did That Made Us Love Her. Your tools are the same. Your goal is the same. Possible differences are that one takes longer (black holes) and one gets more readers (Jennifer Lawrence).

We don’t want to spoil the story for anyone who hasn’t listened to the entire series, but let’s just say you had to check an amazing array of facts on odd topics ranging from astrolabes to horology to the psychology behind tattoos. Nearly every time John McLemore spoke, there was a raging flash flood of statements to check. Can you choose one or two that were the most difficult, and tell us how you resolved those?

I’ll choose two, one sort of trivial and the other sort of serious, but in their way equally difficult to nail down.

1. Adhesives

I’ll try to be vague to avoid spoilers.

After one of the bigger plot developments, Tyler made a kind of sculpture that included a photograph of John B. McLemore. Here is the language, altered to conceal the twist, that S-Town producer and host Brian Reed wrote, and which I fact-checked. (The in-line note is from Brian, to me, saying that he’s not sure the indicated fact is accurate and would I please verify it.)

On top of the sculpture, Tyler [a friend of McLemore] will inlay a piece of stained wood…in which he’s burned John’s name…and also affixed and shellacked CHECK PLS in his favorite photo of John…with John leaning back in a chair, outside, his feet up on a table…

To clarify, the question Brian was posing to me was, Did Tyler use shellac? Or did he use some other adhesive? Brian couldn’t find a reference in the interviews, but he had a dim memory of hearing the word “shellac.”

[pulldata context=”‘Fact-checking is a great ambassador for journalism’ & more from @stownpodcast ‘s Ben Phelan”]

The first thing I did was to look through the interview transcripts, of which there was a great abundance. I found no references to shellac. Rats. So I asked Tyler if he remembered what he’d used in the sculpture. Shellac, polyurethane, lacquer, epoxy…?

When you bear in mind that I was asking Tyler to recall an inconsequential detail from a year and half ago, which he likely spent zero seconds considering even then, you might not be surprised to learn that he didn’t remember what he had used. He told me, sure, it could have been shellac. Yeah, fine, shellac sounds good. So Brian and I talked about it. Could we in good conscience report that the picture was shellacked in place, based on Tyler’s permission and vague quasi-confirmation? No. We could not. So I went back through the transcripts again. This time I found a place where Tyler described making the sculpture, but there were others in the room and he was carrying on two conversations at once. So he said something like, “So, you know, we had…let’s step outside…[unintelligible] that epoxy…but didn’t see…you know, lacquer and polyurethane…but no, I just used it.”

I’d gone to the only person who would know, twice, and he didn’t know. This would seem to be the end of the line. But we needed to say something, and it had to be right, so I called a friend of mine who’s a carpenter and described my dilemma. My friend wouldn’t know what Tyler had used of course, but he might be able to help me figure out what he could have used, based on the materials and so on, and that might help us to frame the question better to Tyler.

We had a long, long conversation. It got theoretical. At the end of it I was no closer to knowing the fact of the matter. My friend apologized and told me that if he could only see the thing he might be able to tell, from its texture or glossiness, what the adhesive was. Well, I had a picture. So I opened the file up and before I emailed it to him I scrolled down to the bottom. There next to the sculpture, on the floor, with a paint brush lying on top of it, was a can of Minwax polyurethane.

So it was polyurethane! It had taken days, but we got to the bottom of it.

Well…the can was closed and we couldn’t see inside. What if it was empty? What if Tyler had opened the can, seen it was empty, then closed it again and reached for the lacquer? The epoxy? The shellac?

Howbeit, we went with polyurethane. I feel pretty good about it. Fact-checking is crazy.

2. Mercury

This contains a bit of a spoiler from chapter seven. Caveat lector.

From the beginning, Brian and producer Julie Snyder knew some disconnected facts about John and mercury. They knew, from interviews and receipts from his clock-repair business and so on, that John had fire-gilded clocks for years and years. They knew that this process required vaporizing mercury with a torch. They knew that inhaling mercury was dangerous and was historically associated with certain mental and emotional disturbances. They knew that John exhibited these behaviors. But they felt they couldn’t report these two facts — the fire-gilding and the possibly related behavior — because it seemed to be irresponsibly speculative to assert, or even to imply, a link.

Compounding the uncertainty, where the scientific literature on mercury poisoning is not old and outmoded it is modern and contentious. Mercury poisoning was a serious problem in the 1800s because a number of industries used it in chemical processes, unaware of (or indifferent to) the fact that it was dangerous to do so. Research into mercury poisoning abounded back then, but the methods of Victorian science are not all that rigorous by the modern standard. On the other hand, modern research is better, but there’s not much of it because people in the developed world don’t inhale large amounts of mercury vapor. So we had a large body of research that we couldn’t rely on because it was bad and a body of good research that we couldn’t rely on because it was small.

The next problem was that there is pretty rowdy disagreement about the effects of mercury toxicity. Some of this is well-motivated, around questions of whether some people are more sensitive than others to mercury, or the possible ill health effects of using dental amalgam to fill cavities. But some of it stems from the discredited and debunked belief that there is a connection between certain vaccines and the onset of autism. (There is no known link.) For those who assert a link, the problematic chemical in the vaccines, the one that they say “causes” autism, is mercury. For all these reasons and more, it can be hard to tell, when you get a mercury expert on the phone, if the science they are describing is legitimate or not.

So we had a large body of research that we couldn’t rely on because it was bad and a body of good research that we couldn’t rely on because it was small.

We only began to gain clarity on the question after I talked to a scientist and a doctor who do fieldwork in developing countries in which a practice very much like fire-gilding still takes place. It’s called artisanal mining, and involves vaporizing mercury in order to extract gold from soil and sediment. These researchers had seen first-hand the effects of inhaling mercury vapor over many years. They were in a position to have a useful opinion. Through them I contacted other scientists and researchers doing similar work.

None of these sources could diagnose John, and none of them could definitively link fire-gilding to his behavior. As Brian points out in the story, lots of people who do not inhale elemental mercury are depressed and anxious. But interviews with these sources and others allowed us confidently to conclude a few things:

• Based on multiple interviews, we knew John had vaporized large amounts of mercury over a long period of time, using no protection or ventilation.
• Our experts told us that, given mercury’s physical properties when it evaporates, it was nigh on impossible to imagine a scenario in which John did not inhale large amounts of mercury vapor.
• Mercury is toxic to multiple body systems, and the historical association with mental illness seems to be real.
• John behaved in ways that people who have inhaled toxic levels of elemental mercury also behave.

We were not able to draw a causal link between mercury inhalation and his behavior. In a sense, these are two islands that don’t touch. But we know they are individually true and that a link is plausible, perhaps even likely. And so that is what we reported.

Can you tell us a little about the fact-checking process for “S-Town”? For instance, how much time did you have to fact-check each episode? Did you visit S-town? Talk to John McLemore personally? Did you have the final say in the fact-checking, or was there another step involved?

I was pulled into the show about a year before it aired, in a research capacity, but that was after it had taken a slightly different direction, based on events depicted in chapter two. (Being vague here to avoid a spoiler.) Which is to say that I wasn’t involved during the initial reporting, when Brian was first making contact with John and Tyler and so on, so I didn’t end up talking to John, though I did listen to a lot of extraneous tape during research and fact-checking. I didn’t get to know John except in the indirect way that the listeners do. But as we were finishing up chapter seven, coming to the end of our time spent in John B. McLemore’s company, I have to say that I started to feel pretty melancholy. And I understand from Brian that that feeling was general around the office, among others who didn’t know John either. People tearing up while they were inserting cross-fades into Pro Tools sound files. And I never did go to Woodstock, though I was on the verge of driving down a couple times, until some intrepid Alabamian freelancers  came to the rescue. Ashley Cleek, Lloyd Bricken, Will Dahlberg—high-five!

Work expands to fill time, but Parkinson’s Law is especially relevant to fact-checking. Because you’re trying to unearth some kind of ultimate truth, there is probably no limit to the amount of time you could usefully spend verifying facts, though at some point I guess you do start to get a diminishing return on your time investment. Brian and I spent entire days on the phone with each other, 10-hour stretches, grinding out the fact-checking and squeezing every fact. Hey, I have to go to the bathroom and make some lunch, don’t hang up, I’ll be back in 20 minutes. So there was enough time to fact-check each chapter, but it’s in the nature of deadlines that you could always use more time. I remember sending an email with the subject line “Urgent astrolabe request,” a phrase that probably hasn’t been articulated since the 15th century. I think we started fact-checking chapter one in November, and released all seven simultaneously at the end of March.

Because Brian is so intensely scrupulous, there were no squabbles between him and me about the fact-checking being too persnickety or obsessive. Brian has said he loves fact-checking, and he does. Good writers welcome any opportunity to revisit their work, as it gives them yet another chance to improve it. So I didn’t have the last word, nor did Brian. The impersonal process did. Which is what you want.

Fact-checkers around the world would be grateful if you could share a couple of your top fact-checking sites/resources. What made your life a little easier during this project?  

Oof. Census.gov? …books? Experts? The long-suffering court clerks of Bibb County? I’m not sure how to answer this intelligently because every question of fact is its own little world, or island. But in my experience, email, for instance, is not as useful as you might expect. It’s good for setting up phone calls. On email, people tend to give incomplete or vague responses when you need clarity and precision — not because they’re being evasive, but because fact-checking is such an inhuman process. If you use email to submit your queries to sources, you have to work around that tendency by breaking every question of fact down into its constituent jots and tittles, and that can be alienating. It can give people the wrong idea, make them paranoid. Why are you asking me five questions about my haircut? Is this story about…my hair? When really you’re only trying to support a throwaway passage like, “Ben’s hair hung in a tangled snarl down to his earlobes. It hadn’t seen a barber since the fall.”

Fact-checking is a great ambassador for journalism, especially with “alternative facts” and “fake news” on so many tongues, and long phone calls can convince people of that. When they realize that you will spend preposterous amounts of time confirming trivia, you win their trust. And you deserve it.

On Wikipedia: A good friend of mine who also does some fact-checking told me that when she needs two sources she just refreshes the Wikipedia page. Joke. But in fact Wikipedia is useful. If you don’t know the first thing about AdS/CFT correspondence or the structure of the Belgian parliament, read the Wikipedia page. It’s pretty accurate, but you have to treat it as a backgrounder. Wikipedia cannot be a source, because you’ll have false confidence and end up trying to support precise statements of fact with a general overview, but it can point you to sources. Go down to the footnotes and follow the links. Read the linked article and studies. Call the authors, ask them whom else you might call, and you’re off to the races. It’s fun. You become an expert.

Finally, the process that they use at This American Life/Serial/S-town is itself helpful, as it involves a lot of annotating and footnoting. We created a paper trail and record of every verification, which kept us from perseverating endlessly on knotty issues, allowing us to move forward. The paper trail is a great solace when you get paranoid in the middle of the night.

We need to make sure you’re human: Did you have to correct any facts, post-production?

Question does not compute. Please rephrase in form of equation.

I am sad to say that I forgot to ask Brian to thank an important source on the air. When I was talking to Darin Hayton, a historian of science at Haverford, about my astrolabe emergency, I was also giving my daughter a bath, so my notes were wet and illegible. We regret the error.