Twitter CEO and Elon Musk clash over bot-battling metrics

Twitter is full of bots, this much we know. But how full, and what kinds of bots? With estimates ranging from Twitter’s own “under 5 percent” to independent researchers suggesting 20 percent or more, it’s clearly a tricky number to nail down, as the company’s CEO, Parag Agrawal, explained in a thread today. Prospective buyer Elon Musk responded with a poo emoji.

Agrawal pointed out that spam and bots are serious problems that all social media platforms contend with, and more importantly they are an evolving and “dynamic” one. “The adversaries, their goals, and tactics evolve constantly — often in response to our work! You can’t build a set of rules to detect spam today, and hope they will still work tomorrow.”

The problem of figuring out whether an account is automated, semi-human, benign, violating, etc is non-trivial yet millions of accounts are actioned in some way, and as on other platforms, usually before they even do anything.

One reason it’s difficult to gauge whether an account is “real” or not, for whatever definition of “real” you choose to apply, is that there’s a limited amount of information available publicly. As Agrawal notes: “The use of private data is particularly important to avoid misclassifying users who are actually real. FirstnameBunchOfNumbers with no profile pic and odd tweets might seem like a bot or spam to you, but behind the scenes we often see multiple indicators that it’s a real person.”

By “private data” he likely means things like direct message activity, logins and browsing behavior that are invisible to anyone viewing from the outside but clear to the internal systems. Many Twitter users engage with the platform silently, and who can blame them?

This is convenient for Twitter because no one can verify the numbers it puts out. Though there’s little reason to think the company is outright fabricating or doctoring the numbers here, it’s inarguable that they have motive and opportunity to do so in subtle ways that would only be visible to an auditor with access to the same data they do.

The question of user authenticity, of course, goes right to the heart of a social media platform’s reach and ability to monetize, and we’ve seen over and over that falsifying or misrepresenting these numbers can have serious effects on the willingness of advertisers and premium services subscribers to pay.

Or, as billionaire and hopeful Twitter owner Elon Musk put it: “💩”

His follow-up question, “So how do advertisers know what they’re getting for their money? This is fundamental to the financial health of Twitter,” is a baffling one. As someone ostensibly interested in running a social media company, it’s difficult to believe that he would not have performed some basic due diligence on the types of metrics that the industry uses to keep track of these things. After all, as Agrawal points out, these numbers have been reported regularly for a long time.

It’s not that the question is a bad one, it’s just odd that he would ask it here and now, after making a very risky buyout offer of the business — a business which he seems to not understand the elementary operations of. Companies like Twitter, Facebook, Snapchat and others that monetize engagement have been defining and redefining “how advertisers know what they’re getting for their money” for a decade.

And long before that, of course, there has always famously been a disconnect between advertising and results — the old, “half works and half doesn’t, but no one knows which half is which” conundrum.

The most pertinent question here does not seem to be “how do we know engagement is authentic?” but rather, why has Elon Musk only begun looking into this now? It’s a bit like buying a horse and then looking up “horse” in the dictionary. The seeming lack of familiarity not just with the complexities of Twitter but with the way the social media ad market and authenticity metrics are defined and handled in general will surely only add to the worries of those who fear Musk is far from the best person to lead the company.