X自建团队清理有毒内容

Under owner Elon Musk, X is bringing some content moderators in-house. Will it help?

JONATHAN RAA/NURPHOTO VIA GETTY IMAGES

In the spring of 2023, when X was still called Twitter, the company began planning a new system to keep the most undesirable content off of its platform.

In place of the army of contract workers that policed most social media sites, including Twitter, the company would build its own, smaller, in-house team of content moderators — a specialized safety net to prevent the most egregious stuff from slipping through without crimping too much on Twitter owner Elon Musk’s outspoken commitment to “free speech.”

Last week, nearly one year later, X announced a new Trust and Safety Center of Excellence in Austin, Texas. The 100-person team of content moderators touted by an X representative in a Bloomberg news report is significantly smaller than the 500-person team that was initially envisioned, according to a former trust and safety staffer. And it’s unclear if X has hired more than a dozen or so people so far.

Still, at a time when lawmakers are turning up the heat on social media companies for endangering children, X’s safety center in Austin has clear PR value. The new center will “bring more agents in house to accelerate our impact,” X CEO Linda Yaccarino said at a Senate hearing on Wednesday.

While some critics took note of the opportunistic timing of the announcement, the details of X’s Austin plan raise a bigger question about the Musk-owned platform: Could Musk’s unconventional approach to content moderation outperform the social media industry’s woeful track record of online safety, or does it represent just another means to cut costs by an organization with little interest in deciding which content is appropriate for its users.

According to several content moderation experts and current or former X insiders that Fortune spoke to, a team of in-house specialists could provide significant advantages compared to the current industry norms. But many also stressed the importance of a coherent underlying policy and investments in tools and technology.

“X+100 is better than just X, in terms of moderation capacity,” a source familiar with trust and safety at X explained. But, the person continued, “the number of humans at computers matters less, in some ways, than having clear policies rooted in proven harm reduction strategies, and the tools and systems necessary to implement those policies at scale — both of which have been dismantled since late 2022.”

X did not respond to requests for an interview or to comment for this story.

Linda Yaccarino, CEO of X, at a Senate hearing on online child safety

ANDREW CABALLERO-REYNOLDS/AFP VIA GETTY IMAGES

Why X decided to bring the content police in-house

The flood of problematic content on X has become a frequent topic of public debate—and disputes— since Musk’s $44 billion acquisition closed in November 2022.

After the Center for Countering Digital Hate published a report claiming that X failed to moderate “extreme hate speech,” Musk sued the group for doing calculated harm with “baseless claims.” Meanwhile videos of graphic animal abuse have spread widely on the platform, according to reports. And just last week explicit, AI-generated content featuring Taylor Swift circulated unchecked for 17 hours until the platform shut down the ability to search for her name at all.

X also leans heavily on its Community Notes feature, which allows approved users to add a note to posts with additional context, to moderate its millions of active users, the former trust and safety staffer said. But the person emphasized that this is merely “one tool” that should be used for moderation. What’s more, a Wired investigation uncovered coordinated efforts within the feature to propagate disinformation, highlighting a lack of significant oversight from the company.

By some estimates Musk has cut 80% of the engineers dedicated to trust and safety and thinned the ranks of outsourced content moderators whose job it is to monitor and remove content that violates the company’s policies. In July, Yaccarino announced to staff that three leaders would oversee various aspects of trust and safety, such as law enforcement operations and threat disruptions, Reuters reported. However, according to another source at X, it’s unclear where T&S stands within the organization’s hierarchy, noting that the group doesn’t appear to be “at the top level anymore.”

However, within Musk’s social media company, there also has been an effort to rethink how the job is done.

Change of plans: Welcome to Austin

The new Austin center actually began as a Bay Area center. The intention was to establish the center in a city like San Francisco, which would help recruit top-tier multilingual talent—crucial for countering Internet trolls, since more than 80% of X’s user base don’t live in the U.S., a source familiar with trust and safety under Musk told Fortune. Given the nuances of individual languages, and the idioms and expressions unique to each one, the idea was that someone familiar with a specific language or culture could, for example, better distinguish a joke from a threat than could a low-paid generalist contract worker with no specialized skills.

“They actually started this by hiring people in the Bay Area to test their quality level and whether or not it would work better than having it outsourced,” the former staffer said. “[X] hired a small team of people and tested it out and their ability to make accurate decisions.” The plan called for starting with 75 staffers and to eventually scale it to a 500-person team if it delivered results.

However, Musk, at that time, leaned towards a more cost-effective location, favoring Austin, because he was certain in his ability to attract and potentially relocate individuals proficient in various languages. The change has added a few wrinkles to the project.

“Having to hire hundreds of people and get them up and running and trained and all that is a roughly two to three month process,” the former X staffer explained. “Then you start training and so you know, realistically you’re looking at three, four or five months before you get a team in place. That assumes the job market is awesome, right? And you don’t have to relocate people and all of that fun stuff.”

According to LinkedIn, a dozen recruits have joined X as “trust and safety agents” in Austin over the last month — and most appeared to have moved from Accenture, a firm that provides content moderation contractors to Internet companies. It’s not clear if X has a contract-to-hire plan in place with Accenture—whereby workers retained by consulting firms like Accenture are given full-time roles at a client company when there’s a good fit—but the source confirmed that X has used Accenture’s contracting services in the past.

The trouble with enforcing rules that are constantly shifting

There are a lot of questions about what exactly the Austin team will focus on. Will they focus on content involving only minors, or only in the U.S.? Will they focus on individual posts, or conduct investigations into sexual exploitation?

“100 people in Austin would be one tiny node in what needs to be a global content moderation network,” former Twitter trust and safety council member Anne Collier told Fortune. “100 people in Austin, I wish them luck”

Whatever their task, social media moderation experts agree that the company will need to make a significant investment in AI tools for the team to be most effective.

Facebook for example employed about 15,000 moderators globally in 2020 when it announced it was “marrying AI and human reviewers to make less total mistakes,” the Verge reported at the time. Snap operates similarly, stating in a blog post that it uses “a combination of automated tools and human review to moderate.”

According to the former X insider, the company has experimented with AI moderation. And Musk’s latest push into artificial intelligence technology through X.AI, a one-year old startup that’s developed its own large language model, could provide a valuable resource for the team of human moderators.

An AI system “can tell you in about roughly three seconds for each of those tweets, whether they’re in policy or out of policy, and by the way, they’re at the accuracy levels about 98% whereas with human moderators, no company has better accuracy level than like 65%,” the source said. “You kind of want to see at the same time in parallel what you can do with AI versus just humans and so I think they’re gonna see what that right balance is.”

But no matter how good the AI tools and the moderators, the performance is only as good as the policy that drives it, and that’s an area where X has struggled under Musk.

Policies need to be flexible enough to adapt to cultural contexts, but they also need to be sufficiently predictable for everyone to understand what the rules are, the source familiar with trust and safety at X explained. This is especially important when moderating content on a large platform “where hundreds or even thousands of moderators have to understand and interpret a stable set of rules. You can’t implement rules consistently and accurately if they’re constantly shifting,” the person said.

The loosening of the rules, and the resulting lack of clarity, has been one constant at X under Musk’s stewardship.

After Musk’s takeover, he went on to reinstate a slate of accounts that were banned for breaking the platform’s policies: Rep. Marjorie Taylor Greene violated COVID-19 misinformation policies, the Babylon Bee posted a transphobic story that violated Twitter’s hateful conduct policy, Andrew Tate (banned from Facebook, Instagram, and TikTok) was banned for saying that women should bear “some responsibility” for being sexually assaulted. All these people were reinstated under Musk.

Some outlets speculated there was a link between the exit of Ella Irwin—the final head of trust and safety during Musk’s tenure—and Musk’s criticism of the team’s decision to remove Matt Walsh’s transphobic “What is a Woman?” documentary, a violation of X’s rules. Despite the violation of X’s written policy, Musk insisted the documentary stay up.

“It’s not obvious to me that X moderates in accordance with policies at all anymore. The site’s rules as published online seem to be a pretextual smokescreen to mask its owner ultimately calling the shots in whatever way he sees it,” the source familiar with X moderation added.

Julie Inman Grant, a former Twitter trust and safety council member who is now suing the company for for lack of transparency over CSAM, is more blunt in her assessment: “You cannot just put your finger back in the dike to stem a tsunami of child sexual expose – or a flood of deepfake porn proliferating the platform,” she said.

“In my experience at Twitter from 2014 to 2016, it took literally years to build this expertise – and it will take much more than that to make a meaningful change to a platform that has become so toxic it is almost unrecognizable.”