How to choose a commenting platform for news sites

Comments can provide readers a voice. Comments can provide a space where readers can contribute new information, like sources sparking new stories, investigations and reports.

Comments can also, among other benefits, increase website traffic for news organizations.

A list of questions to ask and best practices for news organizations seeking return on investment for the commenting platforms they provide.

“ ”

Approximately 70 percent 101 newspaper and online editors/publishers surveyed across the country said they valued comments, according to a recent Associated Press Media Editors survey. That doesn’t mean comments are easy. In his letter to readers announcing The Chicago Sun Times’ temporary suspension of comments, Managing Editor Craig Newman captures the paradox.

“The world of Internet commenting offers a marvelous opportunity for discussion and the exchange of ideas,” wrote Craig Newman. “But as anyone who has ever ventured into a comment thread can attest, these forums too often turn into a morass of negativity, racism, hate speech and general trollish behaviors that detract from the content.”

Recently there has been more movement to improve systems for commenting and reader contributions, such as The Knight Foundation’s investment of $3.89 million to help The New York Times, The Washington Post and Mozilla create an open-source community engagement platform. And there have been many calls to turn off comments.

As The American Press Institute’s first Summer Fellow, I examined these deliberations and took a closer look at what types of comment sections news organizations are using, and what, if any, value they are adding to news organizations’ overarching strategies. I reviewed academic and industry literature, spoke with industry managers and university researchers and conducted a small poll of news organizations across the country. The result is a list of questions to ask and best practices for news organizations seeking return on investment for the commenting platforms they provide.

After researching for two months, I’ve identified these five problems news organizations struggle with regarding their commenting systems.

• I don’t even know where to begin.
• I don’t understand all my options.
• I don’t like the nature of the comments posted on our website.
• I cannot decide how to approach the anonymity and authenticity issues.
• I don’t have time or money.

There is no one-size-fits-all solution. But for the specific problems within these five challenge areas, I’ve identified key questions, considerations and links to further reading for evaluating what commenting strategy works best for you.

Chapter 2

Where to begin

The first question should be this: What is the goal your organization hopes to achieve with its commenting section? Do you want to engage your readers? Build a community? Consider the following.

First, determine your organization’s goal with its commenting section. Gary Graham, editor of the Spokesman-Review in Washington state, admits that “community engagement is a buzz phrase these days.” However, he said that’s the best definition of what his paper is aiming for with its commenting section. “We’re just trying to get people to talk with us,” he said.

To achieve this, the publication recently removed comments on national and wire stories, which attracted polarized unproductive conversations, and moved comments from beneath local stories to a separate discussion page. “We want to encourage more constructive and civil discourse on local issues of importance and interest to readers in our region,” Graham wrote.

For The Washington Post, New York Times and Mozilla project, the most ambitious goal is to “create a feature that would efficiently highlight the most relevant and pertinent reader comments on an article,” Paul Farhi writes.

Next, understand the distinction between an audience and a community. Annmarie Dooling, a community strategist who has worked with The Huffington Post, Salon.com and Yahoo, said that just because a news organization has an audience does not mean it has a community. “An engaged audience and a community are two totally different things,” she said. An engaged audience is “more about one-on-one communication” while a community “revolves around the idea of group think.”

During a July 23 #WJChat with ProPublica’s senior engagement editor Amanda Zamora, tweeters discussed the subject of audience versus community. #WJChat participants attributed more value to communities than audiences, noting that community members participate, engage and are invested and willing to listen, read and contribute.

Zamora tweeted: “You can scale a cheap audience, or develop a more valuable community.”

To develop a valuable community requires engaging with members of it and getting them to engage with each other.

So, what exactly is community engagement?  Community engagement means news organizations are making it a top priority to listen,  join, lead and enable conversation to elevate journalism, writes Steve Buttry.

This means not only affording readers the opportunity to discuss content via comments, but also listening to what they have to say. Engaged readers can introduce new story ideas, point to errors and fix the facts. “I’ve seen a number of examples of significant errors in research being discovered through comments on scientific articles and blogs, something that is arguably similar to the peer-review process,” writes Mathew Ingram for Gigaom.

Do you want to engage your readers or encourage them to take action? “Many organizations cite “engagement,” but what they actually mean is “action,” writes Dooling, who specializes in audience development and social journalism. “They want to motivate their readers to do something, whether that action is clicking a share button, emailing a tip, or contributing some form of user generated content. You want them to spend more time reading, and to have some connection to the work you’re doing, maybe by being a fan of a writer, or signing up for a newsletter.”

To Dooling’s point, commenting sections can support strategic goals outside of community engagement. Commenting sections can also collect new information, provide opportunities for targeted advertising and increase traffic and pageviews with returning users, which was a goal that was ranked important in a small poll I conducted with news organizations.

Where a news organization chooses to host its commenting section can also impact the traffic to its website. “You want reader-to-reader discussion around the topics your organization cares about, and may not want another platform to own those conversations,” Dooling writes.

Chapter 3

Understanding your options

I’ve identified four main types of commenting systems:

• Traditional – threads at the bottom of a story
• Structured comments – requires an extra structural step to comment
• Annotated comments – inline threads
• Stand-alone discussion platforms with user-generated threads

Many of these systems incorporate elements of the others. A news organization might have what appears to be a traditional commenting system, but also include functionalities associated with social media sites, such as including a “share” button, or it could include a voting or ranking feature to sort comments in ways other than by their timestamps.

According to the recent APME survey on comments, which was shared with me for this research, 94 percent of respondents’ news organizations allow readers to post comments at the end of stories and columns posted on their websites.

“For the most part the structure, format, and technical requirements/output of comments have been the same since the start of blogging,” said Aram Zucker-Scharff, content strategist at CFO Publishing, who also works in new media consulting and web development. “We’ve made them faster, real-time, and given them more sign-ins, but DISQUS is perhaps the most ‘advanced’ commenting tool out there and about three features differentiate it from the most drop-dead simple commenting platforms out there. It’s not much.”

Many of us are familiar with and accustomed to seeing this type of “box” –  displayed beneath the article – where readers can input their comments and see other comments in the chronological order they were posted. For the sake of simplicity, I’ll call this a traditional commenting structure.

Dooling, who works in audience development, writes that readers understand their place in this structure is “delegated to a small section beneath piles of ads, videos, banners, and slideshows.” After all, it’s not new. Blogger has been around more than 15 years. “[Readers] feel that no one is listening, and so, they come in angry, flailing wildly, begging to be heard with outrageous statements in caps lock,” writes Dooling.

To cope with this dynamic, some news organizations have moved to third party tools and services, like using a social network plugin. The Huffington Post recently adopted this strategy in June when it transitioned from custom-built comments to using Facebook’s plugin, Comments. Using this commenting service requires that anyone who comments must be a registered Facebook member. Comments are stored and shared on Facebook’s servers.  Facebook comments discourage spam and provide easy interactions on the page.

Using Facebook comments can be a way to drive traffic to your website as well as limit bullying and potentially decrease incivility by eliminating anonymity. However, in eliminating anonymity, news organizations using Facebook comments might also see a decline in participation.

DataUniverse reports that Facebook is the most used service to provide comments to websites, but other commenting system variations also represent a large portion of the marketshare:  Disqus currently holds the number two spot, CommentLuv number three and Livefyre number four.

Comments platform	% websites using
Facebook Comments	52.6%
Disqus	39.2%
Other	8.2%

Data Source: Datanyze, as of September 2014

AMERICAN PRESS INSTITUTE

These variations can cut across platforms, support  “comprehensive comment managing” and often integrate social media and offer customization options. Disqus, the comment hosting service that publications such as ProPublica use, is designed to build communities. Crunchbase describes the service as a “platform for social integration, social networking, user profiles, analytics and more.” Livefyre, which The Seattle Times now uses,  offers a “suite of real-time products” to increase traffic, revenue and user engagement. Livefyre captures “the conversations” about your posts across Facebook and Twitter, to “pull” more people into the conversation.

Over the last couple years, some news organizations are incorporating comments that are less “free for all.” Structured comments add an extra layer in the commenting system to help organize readers’ responses. For example, before the reader can post his comment, he must answer the question prompted. This question could address the reader’s opinion of the article topic; this is how The New York Times and The Washington Post structured comments around their coverage of the Defence of Marriage Act (DOMA) ruling in 2013. In many designs, structured comments help readers and editors see and better understand the overall audience and the viewpoints expressed.

Other publishers like Quartz focused efforts around annotated comments, which are comments appended at the paragraph level rather than  the bottom of the story.

Quartz attributes its idea for annotated comments to the 17th and 18th century newspaper tradition of leaving space in the margins for readers to write down their thoughts and ideas. “Annotations are an extension of that tradition,” according to Quartz’s website. The idea behind this system is that readers can input their thoughts directly into the relevant parts of the content, which can help keep related comments together.

A critique of this type of system, and similar systems such as Genius.com (previously Rap Genius), is that the annotations can become distracting, especially when the platform is relatively new. Quartz hides the text of the annotations themselves and uses a subtle icon to signal that a paragraph has an annotation available.

Other news sites have built stand-alone discussion platforms with user-generated threads, such as Gawker’s Kinja, which Gawker says allows readers “the power to curate the conversation” by using the same tools and their functionalities — replying, dismissing, following, liking and sharing —  as editors.

Chapter 4

How to shape the nature of comments posted

Deciding how a commenting system looks and operates are choices publishers should make with intent. These decisions affect the nature of readers’ and moderators’ experiences and the content of their contributions.

Talia Stroud, associate professor of communication studies at the University of Texas – Austin, studies how the media affects our political behaviors and attitudes and how those attitudes and behaviors affect people’s use of media. Before deciding what type of commenting system to go with, Stroud said to consider:

• The type of comments you would like to see on your website
• The culture of your publication and audience
• The design components

Consider how these three factors could work together to create a unique participatory environment for your readers. If you create a system that incentivizes a certain type of comment by elevating them, like “NYT Picks” or Gawker’s Kinja stars, a community can rally around the system to drive engagement and discussion value. “These incentives could be very different depending on the newsroom,” she said.

Tommy Craggs, who is the editor-in-chief of Gawker’s sports website, Deadspin, said that this “discussion” can take on multiple forms. ” A joke followed by a round of virtual applause is, as far as Kinja is concerned, a lively discussion.So is a well-informed exchange about hearsay exceptions,” wrote Craggs when the platform was introduced in 2012.

The New York Times selects the “most interesting and thoughtful” comments, expressing a variety of views, to be featured prominently in its “NYT Picks.” These may also be chosen to highlight discussions from a certain region or readers with “first-hand knowledge of an issue.”

What are some incentives news organizations could provide to encourage desirable comments?

Incentives could be in the form of voting, ranking, highlighting, or even through money.

Quartz rewards “great annotations” by both responding to these annotations and displaying them “more prominently.” On the flipside, the news organization removes others that are “off topic or abusive.”

Cory Blair sought to understand and learn about the Kinja’s audience through “becoming an active member of it.” In his quest to become a “Kinja star,” Blair wrote that Gawker thought carefully about what types of conversations it wanted to take place on Kinja, including the motivations and incentives Gawker uses to encourage “quality content” on the platform. Great Kinja commenters can even be contacted to write for pay through the company’s “Recruits” program.

In announcing the Recruits program, Gawker’s editorial director Joel Johnson wrote that some of the site’s “best writers” rose up through “commentariat,” but that not every commenter is fit for what is required in “real reporting.” And, with this, the idea for Recruits was born.

Recruits are paid, short-term contractors. They receive a stipend, and their own “Kinja subsite attached to one of eight core brands, mentoring and direction from our site leads and their staff, as well as an aggressive bonus structure that will reward the capturing of uniques (using the same Quantcast ‘people’ metric we use across all the sites),” according to Johnson.

How do I get rid of abusive and off-topic comments?

Sometimes quality can get lost in traditional commenting systems as they are posted chronologically and can be overrun with incivility.

More advanced systems like Gawker’s Kinja have also played host to incivility. Gawker’s “weakness” became especially clear this summer when one of its publications, Jezebel, became bombarded by obscene pornography and violent trolls.  Consequentially, Jezebel had to turn off the upload ability and shut off comments on select articles, and Gawker disabled photo comments. As of the publication of this report, no long-term fix for the issue has been decided.

Some organizations have opted to get rid of comments. Last year, Popular Science chose this course, citing a University of Wisconsin-Madison study that found that uncivil comments polarized readers and “often changed a participant’s interpretation of the news story itself.”

Others approach this issue by adding a layer of structure into the commenting process.  Another option is only allowing comments occasionally, or on select articles. The New York Times allows commenting on 17 articles a day, which are selected based upon four factors: the story’s news value, predicted interest, staff’s ability to moderate the estimated volume of comments in a “timely fashion” and whether an article on the same topic was recently open for comments.

Additional features can also assist in filtering out low-quality comments, such as voting, ranking, starring, or liking, which helps create a space where “the good stuff stays at the top and the bad stuff recedes,” writes Bob Cohn, publisher and chief operating officer of The Atlantic. The Atlantic also utilizes its most faithful readers to help moderate comments.

There are some additional variables to consider, such as your time and capacity to moderate and engage with commenters and whether you would like to “own” those comments.

Are your reporters willing to invest a little bit more time? Do they have the capacity?

Talia Stroud and her fellow researchers at the Engaging News Project, which aims to “provide research based techniques for engaging online audiences in commercially viable and democratically beneficial ways” studied: What happens when journalists take a more active role in comments sections?

They teamed up with a local news network in a top-50 market to study what impact a reporter’s interaction,  a station’s interaction and no interaction would have on the TV station’s Facebook community. The study found that when a reporter engaged readers in the comments section — by  replying to questions, asking direct and open-ended questions, providing additional information and recognizing and highlighting good discussions — incivility decreased. Specifically, the study found:

• The chances of uncivil comments declined by 15 percent when a reporter interacted with a post as compared to when no one interacted with a post.
• When a reporter posed a question that only had a few options, such as a yes or no, chances of uncivil comments to a post declined 9 percent in comparison to posts that included a question or those with an invitation for readers to comment.

Though the task of engaging in comments sections may seem “onerous” or another to-do item, the reporter in this study did not expend “extraordinary efforts,”commenting on average four times.

Are you willing to move your comments over to Facebook or another free third-party service?

In a comparative analysis of incivility in online political discussion, Ian Rowe, Ph.D candidate and assistant lecturer at the University of Kent, compared the incivility of comments on The Washington Post’s website to the Post’s Facebook page.  (Note: the Post does not use the Facebook Comments plugin so this study refers specifically to the Post’s Facebook page.) Rowe found uncivil commenting behavior to be significantly more common on the website version of the Post than on the Post’s Facebook page. He hypothesizes this is because on the website, users can remain anonymous, whereas on Facebook, “commenters , are identified with, and accountable for, the content they produce.” This introduces one of the major problems with comments, which is tied into issues of anonymity. There are pros and cons to requiring commenters identify themselves.

Rowe also concluded that the poor behavior in the discussion was more directly targeted toward others when the exchanges occurred  on The Post’s website. The conversations on The Post’s Facebook page were “less likely to be interpersonal, and more likely to be aimed at individuals not involved in the discussion, or used as a way to articulate an argument, rather than offend others.”

Depending on the size of your organization and your goals, Facebook Comments, or another third party platform may be what you’re looking for.

In APME’s recent survey, nearly 57 percent answered that their news organization uses a commenting hosting service. Of these, nearly 62 percent use Facebook Comments as their commenting service.

However, before handing your ownership over, consider the following: Do you want to increase traffic on your page?

Using Facebook Comments can be a way to drive traffic to your website. If a user comments on an article using Facebook Comments, he has the option to share this comment on his Facebook feed. If he chooses to share his comment on his own feed, any of his friends may comment on this. This new comment, made and posted on Facebook.com, can also be seen on the original article’s website.

However, Zucker-Scharff writes that third party sites can exclude you from SEO opportunities too. Using the Facebook “Comments” plugin means that your comments, and all the “data usage” that comes with them, are shared and stored on Facebook’s servers. This alarmed some in the industry, like Dan Gillmor, director of the Knight centre for digital media entrepreneurship at Arizona State.  “The billion-user social network has emerged as the news business’ most potent competitor, capturing attention and advertising on the path to monopoly status,” Gillmore wrote in The Guardian. “Ceding such a crucial part of one’s online operation to a competitor has always seemed insane to me.”

Zucker-Scharff wrote what troubles him the most about using Facebook Comments is giving up control and ownership of these comments on your site. “Even worse, it seems you will never be able to take that control back.” And, what if the system were to “suddenly disappear?” he asks. (It should be noted that DISQUS does let you own the comments, and data, posted through its portal.)

This is one of the reasons that Facebook did not work as a commenting platform for The Seattle Times. “Having Facebook as your commenting system means you have no control over the data and the functionality,” Payne said. “They own those comments, you don’t.”

Payne said the Times installed Facebook commenting as a test on one of the paper’s busiest local-news blogs a couple years ago. Payne said Facebook didn’t work for them because the “the volume of comments went down substantially,” which he said was because people either didn’t trust Facebook, or they didn’t want their real names to appear alongside their comments.

“When you have less volume, you have less constructive dialogue and more drive-by comments,” Payne said and added that the paper still had to ban commenters who broke the rules on Facebook, attacked other commenters and “trolled using their real names.”

More volume in the social sphere doesn’t necessarily imply that you’re building a community. “Social media, and particularly viral successes, seems like a much better way to build an audience — on paper,” said Dooling.

Dooling points out that in her experience, commenters spend more time on-site and visit their favorite news organization’s website more times a day. “They are also much more vocal, which is great for feedback on your website and very different than a social media reader, who may stop by because they were coerced by a snappy headline, and don’t have an invested knowledge in your news,” she said.

Chapter 5

How to handle anonymity and authenticity

Have you ever wondered if there is value in commenters identifying themselves? Or do you struggle with verifying the facts and stories your commenters post? How can we authenticate what they say if we don’t know who they are?

In the previous section, I highlighted research showing commenters are more civil when commenting on Facebook and less civil when their comments are anonymous. Many websites have started requiring readers to register if they wish to comment.

In APME’s survey, news organizations that accepted anonymous comments and those that did not were about split, with the former at nearly 46 percent and the latter at nearly 54 percent.

Research suggests linking a commenter to his social media profile can cut down on incivility. Is this the solution, or a “quick fix,” asks Dooling, a community strategist who has worked with The Huffington Post, Salon.com and Yahoo.

“The better thing to do is to verify what someone knows, not the name someone goes by. Our fear of anonymity is an extension of our fear of the unknown,” she wrote. “To enable real names for all commenters, you leave your civil readers open to cyber-bullying on multiple outlets, and real-life danger and you’ve mistakenly created a dangerous and antagonizing forum.”

Dooling discusses what Reddit’s “r/science” subreddit forum did to fix the “spread of misinformation” being posted, while still allowing its commenters to remain anonymous. r/science raises comments posted by scientists and known researchers to highlight meaningful and informed discussions.  A member must have a degree in one of the related fields of study and must send moderators a photo of their diploma, a verifiable email address or a business card. All proof is only seen by moderators, and it is never made public.

However, as discussed earlier, anonymity can leave websites subject to trolls. Additionally, as Popular Science noted, studies have shown comments can influence readers’ perception of the news.

In a recent experiment, Kevin Wallsten, associate political science professor at California State University, and Melinda Tarsi, assistant political science professor at Bridgewater State University,  found that Internet users became “significantly more negative” toward USA Today when “exposed to a story” with an anonymous comments section.” Furthermore, they found that it did not matter whether the comments praised or criticized the media, the negativity remained.

Chapter 6

How to manage resources you devote to comments

Before spending too much money, take some time. Spend the time strategizing, researching, understanding what your audience wants, what you want and how comments will help you deliver this.

Can you accomplish this through comments? If you believe so, do you have the time to maintain the system?

The time news organizations are spending monitoring comments is all over the map, according to APME’s survey results.

Hours spent managing comments	% of publishers
2 to 4	15%
5 to 8	11.25%
9 to 12	20%
13 to 16	27.5%
17 to 20	23.75%
20 to 24	2.5%

Data Source: APME survey, 2014

AMERICAN PRESS INSTITUTE

“It seems to me that news organizations create comments sections because they think that they need a space to hear what their readers think about their stories, or to allow ‘free speech,'” said Dooling, audience development specialist. “But many don’t want to put in the time to moderate or keep these sections safe, and some don’t even have community managers.”

Community manager salaries vary, some as low as $35,000, others as high as $85,000. These costs don’t even include the development and maintenance costs of the tools, said Dooling.

And, building a platform like Gawker’s Kinja takes a lot of money. As Nick Denton, Gawker’s founder, told Mathew Ingram, he has the “luxury of being able to experiment with such features because the company is privately held and ‘significantly profitable.'” Kinja was financed 100 percent through Gawker’s operating cash flow.

If cash flow is a problem, and you don’t have time or money, consider doing what you can with what you have and spending more time than money.

Are there other ways to achieve your goal?

It all comes down to your news organization’s goal and strategy: What do you want to achieve? If your answer is engagement, are comments the best route for your organization?

Consider alternative ways to achieve the value you are looking for, such as this list of community engagement techniques, which Steve Buttry put together. Using a different technique could potentially save you money and time.

• Social media
• Blog networks
• Crowdsourcing
• Breaking news
• Engaging through stories and community events
• Curation and aggregation
• Content submitted by users
• Make content engaging
• Voting
• Contests
• Face-to-face
• Schools
• Community groups and feedback

If you’ve already opted not to make major technical changes, consider these options

Updating your policy: The Spokesman-Review in Washington State changed its commenting policy in August 2014. “We no longer will allow comments to be posted on national or international stories, or letters to the editor,” wrote editor Gary Graham, noting that the comments will be allowed on local stories, staff blogs and staff columns, but that these discussions will no longer take place beneath the content. Instead readers now click the link provided where they are brought to a separate page for discussion.

Graham said the two goals behind these changes were to “encourage more constructive and civil discourse on local issues” and to reduce the amount of time staff spend monitoring comments. “It’s no secret that our newsroom ranks are much smaller in the wake of the economic tsunami that has wreaked havoc on the industry, and time spent moderating comments is time we cannot spend on research, reporting and editing,” he wrote.

Using untapped, available resources – your readers

Ta-Nehisi Coates, who has been praised for his work in moderating comments and fostering engagement, said at one point he was spending more time moderating comments than he was writing for The Atlantic, according to Bob Cohn, president and chief operating officer of the publication.

“So if you believe — as I do — that comment sections can be good when aggressively moderated, but you don’t — as we don’t — have the human resources to undertake that moderating, what’s the answer?” Cohn wrote.

Coates identified readers who were “‘wise’ and who had already ‘done the work of moderating by cooling down threads that were on the cusp of becoming knife fights.'” The Atlantic incorporated a new approach by assigning these “faithful readers” to moderate Coates’ comments. “They have the power to discipline and even ban.”

Minimal software updates and design changes

Discuss with your IT staff and see what plugins or updates are available and what changes could be carried out with minimal expense and labor. Graham said the changes recently made at The Spokesman-Review were relatively easy.

Additionally, Stroud and fellow researchers investigated the “like” button because “like” does not always seem appropriate. “Word choices are consequential,” she said.

Stroud’s study, which gathered data from more than 700 people, found that from a business angle, “respondents seeing a ‘Respect’ button clicked on more comments in a comment section.” And, from a democratic angle, “respondents seeing a ‘Respect’ button clicked on more comments from another political perspective in comparison to the ‘Recommend’ or ‘Like’ buttons.”

If you have decided to change your system, consider the following

Free third-party tools and services, such as Facebook or DISQUS

Payne said he believes larger news organizations have an obligation to “curate the discussion” through “moderation and monitoring.” And, although he believes Facebook is a “copout” for larger news organizations because “they see it as a way to not have to monitor and moderate the discussions,” he said the site could potentially serve the needs of a smaller news organization. This is because, Payne said, with smaller news organizations,  “the volume of comments is so low to begin with, and they are less likely to have the resources to build a system or buy a vendor product.”

Don’t forget to ask questions and consider any potential associated costs that come with moving platforms:

• Will you maintain ownership rights?
• Is it customizable?
• Does the platform offer analytics?

Also, look into open-source projects, on websites like Github, and take advantage of free code.

Ask for help

Consider partnering with a local academic institution to test strategies or build a system. The Sacramento Bee partnered with Stanford’s journalism and computer science graduate students to research data and to build a commenting system.

“I was really apprehensive the first time I reached out to USC and Stanford. But, even in the computer science departments of these schools, they understand newspapers serve an important role in their communities and that publishers are struggling and need some help,” said Tom Negrete, The Bee’s director of innovation and news operations, in a Q&A with API.

“You do need someone at your organization who is given some time to talk, explore possibilities and determine what relationship and project your publication should be cultivating, working to make happen,” he said.