阻止网络暴民，人工智能可以做到么？

你有没有在社交媒体被网络暴民围攻过？我有过。去年12月我发了一条推特讽刺白人优越论者大卫·杜克，结果他的支持者一拥而上，把我的推特生生变成了肮脏的下水道，充斥着纳粹般疯狂又难听的人身攻击言论，而且持续了好几天。

没人能战胜互联网上的暴民。只要有点仇恨和谩骂的火种，社交媒体上的人们只能弃械投降，网站也只好撤下评论功能。谁想生活在全是马屁精和疯子的网络社区里呢？

幸运的是，现在下悲观结论可能为时过早。一项新策略有望解决暴民问题，恢复互联网文明讨论的氛围。这个项目由谷歌母公司Alphabet旗下的智库Jigsaw主导，主要依靠人工智能手段，可以解决以往无法审核海量评论的头疼问题。

为了解释Jigsaw的具体做法，首席研究科学家卢卡斯·迪克森将网络暴民问题与所谓的拒绝服务攻击相比较，拒绝服务供给是指攻击者故意用垃圾信息淹没网站，导致服务器过载最后下线。

“网络暴民的区别是不会用垃圾信息攻击网站，而是攻击评论区或是社交媒体账户或话题标签，结果是其他人一句话都插不上，暴民掌握全部话语权。”迪克森表示。

大量恶意评论不仅会对个人造成困扰，对媒体公司和零售商也是威胁，因为现在很多商业模式都围绕着网络社区展开。Jigsaw研究网络暴民时，已经开始量化损失。举个例子，如果有维基百科（Wikipedia）的编辑受到人身攻击，Jigsaw会测算其后该编辑在维基百科上贡献词条频率与受攻击之间的关系。

要解决当前扭曲的在线讨论氛围，根源还在于海量数据和深度学习，这也是人工智能领域发展迅速的一块，主要目标是模仿人体大脑的神经网络。近来深度学习已经在谷歌翻译工具上实现了了不起的突破。

说到评论，Jigsaw让机器学习《纽约时报》（New York Times）和维基百科里的上千万条评论，学会识别言辞中的攻击性以及文不对题的发帖。直接影响是：《纽约时报》之类的网站之前只有能力处理10%的文章评论，但在采用新算法后可以实现100%覆盖。

虽然每家媒体评论区的调性和词汇差别可能很大，但Jigsaw表示可以调整审核工具，适用各种网站。这就意味着即便是小博主或网络零售商，也能放心放开评论功能，不用担心被网络暴民攻陷。

技术爱好者都很关注Jigsaw的动向。最近，《连线》杂志（Wired）上的一篇文章将Jigsaw的新项目称为“互联网正义联盟”，还夸赞了谷歌旗下一系列行善的项目。

但也有些专家表示，Jigsaw团队可能低估了问题的难度。

最近比较高调的机器学习项目主要关注点在图片识别和翻译文本上。但互联网的对话经常很看语境：举例来说，很明显应该让机器学习项目从所有评论中屏蔽“贱货”，但有时人们用到这个词并无恶意，却同样会被算法屏蔽，比如有人会说“生活就像个贱货。”或“其实我本来不想抱怨工作的，但是……”想教会机器从模糊的语境中辨清真实意思其实并不容易。

“机器学习能学会语言规范，但没法理解文字背后的语境和感情，尤其是像推特这么简短的文字。这是人类终其一生才能学会的东西。”前谷歌软件工程师大卫·奥尔巴哈表示。他补充说，Jigsaw的项目可以向《纽约时报》之类的网站提供更好的审核工具，但到了推特和Reddit等更自由的论坛，能发挥的作用就不大了。

种种质疑并未让Jigsaw的迪克森退缩。他指出，网络暴民跟拒绝服务攻击一样都是永远无法彻底解决的问题，但其影响是可以减弱的。迪克森相信，Jigsaw利用机器学习技术方面的最新成果可以控制网络暴民的威力，让和平讨论重获优势。

Jigsaw的研究人员还指出，看起来像暴民团伙的攻击——即突然跳出来一起骂脏话的一群人经常是个人行为，有时是某些组织设置的自动程序模仿暴民团伙。Jigsaw的识别工具正飞速学习迅速识别并阻止这些行为。

此外，有人质疑道高一尺魔高一丈，网络暴民会根据审核工具的特点调整谩骂方式，从而避开屏蔽，迪克森对此也有解释。

“审核工具越多，攻击的花招必然也会越多，”迪克森表示。“理想情况是攻击方式花哨到没人看得懂，没人能懂也就没效果，那么攻击自然会停止。”

那些被社交媒体暴民赶走的人们

2015年到2016年

从NPR到路透，越来越多的大众媒体网站和博客关停了评论功能。

Have you ever been attacked by trolls on social media? I have. In December a mocking tweet from white supremacist David Duke led his supporters to turn my Twitter account into an unholy sewer of Nazi ravings and disturbing personal abuse. It went on for days.

We’re losing the Internet war with the trolls. Faced with a torrent of hate and abuse, people are giving up on social media, and websites are removing comment features. Who wants to be part of an online community ruled by creeps and crazies?

Fortunately, this pessimism may be premature. A new strategy promises to tame the trolls and reinvigorate civil discussion on the Internet. Hatched by Jigsaw, an in-house think tank at Google’s parent company, Alphabet, the tool relies on artificial intelligence and could solve the once-impossible task of vetting floods of online comments.

To explain what Jigsaw is up against, chief research scientist Lucas Dixon compares the troll problem to so-called denial-of-service attacks in which attackers flood a website with garbage traffic in order to knock it off-line.

“Instead of flooding your website with traffic, it’s flooding the comment section or your social media or hashtag so that no one else can have a word, and basically control the conversation.” says Dixon.

Such surges of toxic comments are a threat not only to individuals, but also to media companies and retailers—many of whose business models revolve around online communities. As part of its research on trolls, Jigsaw is beginning to quantify the damage they do. In the case of Wikipedia, for instance, Jigsaw can measure the correlation between a personal attack on a Wikipedia editor and the subsequent frequency the editor will contribute to the site in the future.

The solution to today’s derailed online discourse lies in reams of data and deep learning, a fast-evolving subset of artificial intelligence that mimics the neural networks of the brain. Deep learning gave rise to recent and remarkable breakthroughs in Google’s translation tools。

In the case of comments, Jigsaw is using millions of comments from the New York Times and Wikipedia to train machines to recognize traits like aggression and irrelevancy. The implication: A site like the Times, which has the resources to moderate only about 10% of its articles for comments, could soon deploy algorithms to expand those efforts 10-fold.

While the tone and vocabulary on one media outlet comment section may be radically different from another’s, Jigsaw says it will be able to adapt its tools for use across a wide variety of websites. In practice, this means a small blog or online retailer will be able to turn on comments without fear of turning a site into a vortex of trolls.

Technophiles seem keen on what Jigsaw is doing. A recent Wired feature dubbed the unit the “Internet Justice League” and praised its range of do-gooder projects.

But some experts say that the Jigsaw team may be underestimating the challenge.

Recent high-profile machine learning projects focused on identifying images and translating text. But Internet conversations are highly contextual: While it might seem obvious, for example, to train a machine learning program to purge the word “bitch” from any online comment, the same algorithm might also flag posts in which people are using the term more innocuously—as in, “Life’s a bitch.” or “I hate to bitch about my job, but …” Teaching a computer to reliably catch the slur won’t be easy.

“Machine learning can understand style but not context or emotion behind a written statement, especially something as short as a tweet. This is stuff it takes a human a lifetime to learn.” says David Auerbach, a former Google software engineer. He adds that the Jigsaw initiative will lead to better moderation tools for sites like the New York Times but will fall short when it comes to more freewheeling forums like Twitter and Reddit.

Such skepticism doesn’t faze Jigsaw’s Dixon. He points out that, like denial-of-service attacks, trolls are a problem that will never be solved but their effect can be mitigated. Using the recent leaps in machine learning technology, Jigsaw will tame the trolls enough to let civility regain the upper hand, Dixon believes.

Jigsaw researchers also point out that gangs of trolls—the sort that pop up and spew vile comments en masse—are often a single individual or organization deploying bots to imitate a mob. And Jigsaw’s tools are rapidly growing adept at identifying and stifling such tactics.

Dixon also has an answer to the argument that taming trolls won’t work because the trolls will simply adapt their insults whenever a moderating tool catches on to them.

“The more we introduce tools, the more creative the attacks will be,” Dixon says. “The dream is the attacks at some level get so creative no one understands them anymore and they stop being attacks.” 

***

Driven from social media by trolls

2015–16

Increasingly, popular media sites and blogs, from NPR to Reuters, are eliminating comments from their pages.

2015年7月

Reddit爆发了一场用临时首席执行官爱伦·鲍的话说“史上最严重的网络暴民攻击”，随后爱伦·鲍宣布辞职。

July 2015

Ellen Pao, interim CEO of Reddit, resigns in the wake of what she calls “one of the largest trolling attacks in history.”

2016年7月

网络暴民在电影演员莱斯利·琼斯的推特账号下发了一大堆种族歧视和色情图片，之后莱斯利宣布退出推特。她在最后几条推文里写道，“你们想象不到有多么恶毒。”

本文另一版本刊发于2017年2月1日出版的《财富》杂志上，标题为《网络暴民猎手》。 （财富中文网）

译者：夏林

July 2016

Movie actress Leslie Jones quits Twitter after trolls send a barrage of racist and sexual images. In one of her final tweets, she writes, “You won’t believe the evil.”

***

A version of this article appears in the February 1, 2017 issue of Fortune with the headline "Troll Hunters."