Google’s New Perspective Project Filters Online Comments Based on Toxicity

photo credit: Martino Pietropoli

Online harassment is a hot topic right now, as Twitter’s perennial battle with trolls heats up, forcing the company to develop new features to combat abuse. Technology companies are scrambling to create solutions that will make their communities safer for users and now Google is taking on the challenge of online harassment as part of its Jigsaw technology incubator. Jigsaw’s engineers and researchers tackle geopolitical problems like attacks on free speech, injustice, corruption, and violent extremism.

Perspective is Jigsaw’s latest project aimed at improving the comment sections of websites, which can become hotbeds of harassment when left unmoderated. It turns out that eliminating the darker aspects of human behavior, especially when combatting those operating under the cloak of online anonymity, has proven to be an exquisite challenge best suited to the bots.

The Perspective project uses machine learning to identify and filter comments for toxicity. Its API scores comments based on “the perceived impact a comment might have on a conversation.” Publishers can then use that information to offer real-time feedback to commenters and speed up moderation. The live demo allows readers to filter the comments based on a sliding scale of toxicity that they are willing to engage.

The Perspective site also includes a Writing Demo that delivers real-time feedback for the toxicity level as you type. The model defines toxic as “a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion.”

Developers Can Request Access to the Perspective API and Major Publications are Already Experimenting with It

Toxic commenting and trolls are especially rampant on news sites, requiring moderators to be constantly vigilant and ready to neutralize threats to civil discourse. This is why the New York Times employs 14 full-time moderators to manually review the 11,000 comments that come in each day. Despite the efforts of this dedicated team, commenting is only available on 10% of Times articles due to the moderation load.

As a partner on this project, the New York Times open sourced 10 years of moderated comment archives to help the Jigsaw team build the machine learning models that will improve conversations on the web. The publication is currently creating an open source moderation tool to expand community discussion to other areas of the Times.

The Wikimedia Foundation is also collaborating with Jigsaw to develop tools for automating detection of toxic comments and analyzing their impact in discussions at scale. These tools are aimed at mitigating the personal attacks levied at volunteer editors in an effort to improve overall community health.

The Perspective project is still in its early days of research and development, but developers can sign up to request an API key. Google will be open sourcing the experiments, models, and research data gained from testing machine learning as a tool for improving online discussion.

As WordPress powers more than 27% of all websites, a plugin built with the Perspective API could have a major impact on raising the standard of discourse for a large segment of comment-enabled sites. Many publications that might otherwise value thoughtful discussion, have resorted to turning comments off entirely because of the burden of moderation.

Those who rattle off the tired internet maxim that says “Never read the comments” speak to the pervasive toxicity that has invaded online discourse, but they also betray their own fragility in engaging commenters who sabotage discussions with incivility. Readers don’t always have the emotional energy to deal with rude comments that slipped through moderation. While some may find Reddit-style wild west commenting to be spirited and amusing, there are plenty of others who find it demoralizing.

One thing I appreciate about the Perspective project’s demo is that its aim isn’t to edit or change the comments to be less toxic but rather it offers the reader a way to filter based on the individual’s comfort level. With clear warnings in place and a default view set on the safer side, the publication is no longer obligated to overly-censor comments for the lowest threshold of offense.

The Perspective project is experimenting with using machine learning to wrangle the human factor of interacting online. It’s come to the point that moderating comments and weeding out toxicity has become overwhelming for those trying to run a publication. The most encouraging aspect of this experiment is that Google put engineers, designers, and researchers on this problem because comments still matter. This new technology affirms the importance of public discourse on the web and aims to preserve comments as a safe place for conversations. I’m interested to see what WordPress developers can build with the Perspective API once it is available.

Would you like to write for WP Tavern? We are always accepting guest posts from the community and are looking for new contributors. Get in touch with us and let's discuss your ideas.

19 Comments


  1. Hopefully the API is not based on input from the likes of Politifact, Snopes, Daily Kos, Huffington Post, CNN, etc…

    At the end of the day, an API is just code, written by biased humans, which is what Facebook’s “fake news” controversy reminded us of after the recent U.S. presidential election.

    Like Sarah’s “Why Comments Still Matter” post in 2014 addressed, sites like CNN.com or Popular Science really just disabled comments because they were tired of being called out on their facts (one of the best blog post’s I’ve seen on the topic, btw).

    TL;DR: I doubt this API changes anything, unless it’s just another filter against foul language, or is somehow configurable.

    Report


    1. Strange that your list is missing Breitbart, Fox News, InfoWars and the like. I wonder why that could be?

      Report


    2. Fox News and CNN are exactly the same because both are political party propaganda. Doesn’t matter which side, same goal: keep Americans divided. Anyone still watching TV news should find an independent news source online rather than fill your head with propaganda.

      Report


  2. Readers don’t always have the emotional energy to deal with rude comments

    Sigh. No different than TV shows or movies…if you don’t like it then change the channel. But thanks NY Times and Google for the continued efforts at mind control with the pretense of helping the vulnerable among us…

    Report


  3. Let me make a wild guess here – conservative political opinions will be branded as ‘toxic’.

    Report


    1. Sarah links to a live demo. Want to throw some experiments through it and report back to us?

      Report


  4. This sounds like an interesting project, Sarah. Given the Washington Post has partnered with Mozilla and The New York Times are funding a $4 million project to bring easier comments and photos to news sites, it’s clear publishers still see the value of comments.

    Among many of the things I wrote about in a recent post on the value of comments was the fact that as well as keeping discussions open from both sides of a perspective, comments can also greatly help revenue and ROI (there’s a study that showed commenters are 5x more likely to buy than regular subscribers, for example).

    If tech giants like Google, and publishing giants like TWP and NYT can bring open yet respectful commenting as standard, the web might just become fun again (and profitable for publishers).

    Report


  5. This is a hard nut to crack and i’m glad to see Google putting some weight behind it. I’m especially keen that it is an open api so we in the WordPress world can leverage it.

    There is another player in this space that already has a WordPress plugin available: Sift Ninja. They just came out of beta and while new to the WordPress scene, they come to it after years of experience and data collection doing work for Twitch, X-box, etc. I’ve toyed with it enough to get behind it as the best solution there is for now.

    Report


    1. Interesting – is sift compatible with either Postmatic or Replyable?
      Checking it out now, thanks for the reco.

      Report


      1. Totally. It fits right into the moderation flow without trouble. It’s pretty sweet.

        Report


      2. Cool not heard of Sift Ninja before.

        Report


  6. conservative political opinions will be branded as ‘toxic’.

    Yes.

    Did you know that “Conservative” can be different in each country?

    Liberals in Australia would be considered Conservatives in Canada.
    Liberals in US would be considered NDP in Canada? (In Canada: NDP is left of Centre, Liberals are Centrist and Conservatives are right of Centre).

    Facebook filtered/censored many Conservative content during the US election. well known fact.

    Report


  7. Interesting. The project is called “Perspective”, so I wonder what perspective the project will take when learning and filtering “toxic” comments?

    Report


  8. Some of those issues where why the IMDB boards closed down.

    Report


  9. This type of technology can be extremely useful for moderating/filtering people who are just being trolls or it can be extremely dangerous to the freedom of expression.

    My biggest fear is the growing idea that people have a right not to be offended, which will continually invade technological advances. Freedom of expression is useless if no one is offended and everyone agrees on everything.

    Report


    1. Totally agree. Some people are so easily offended or triggered. Another issue is tolerance. Some people are only tolerant of those who agree with them. How can technology solve these issues?

      Report


  10. Two recent stories stand out in my memory:

    Facebook Apologizes for Censoring Christian Mom

    “Facebook finally apologized and reinstated Johnson’s account after coming under heavy fire by the media.” – that’s what Mr. Bean was talking about… those “lucky” cases where publicity helped draw the attention…. what about those other not so lucky cases?

    Or: Google blacklists Natural News site.

    I hope those BIG guys will not abuse the tools they are creating.

    Report

Comments are closed.