X’s AI chatbot Grok made up a fake trending headline about Iran attacking Israel

A shocking story was promoted on the “front page” or main feed of Elon Musk’s X on Thursday:

“Iran Strikes Tel Aviv with Heavy Missiles,” read the headline. 

Captured trending on X on April 4, 2024.
Credit: Mashable screenshot

This would certainly be a worrying world news development. Earlier that week, Israel had conducted an airstrike on Iran’s embassy in Syria, killing two generals as well as other officers. Retaliation from Iran seemed like a plausible occurrence.

But, there was one major problem: Iran did not attack Israel. The headline was fake.

Even more concerning, the fake headline was apparently generated by X’s own official AI chatbot, Grok, and then promoted by X’s trending news product, Explore, on the very first day of an updated version of the feature.

Tweet may have been deleted

How Musk let this happen

A few years before Elon Musk acquired the platform then-known as Twitter, the company rolled out a new feature that provided written context for trending topics. The technology powering the written trend descriptions? Humans.

Previously when a story would break, Twitter’s algorithms would catch the relevant keywords being posted and share the trend, but it sometimes wouldn’t be obvious why a given set of words was trending.

But, in 2020, Twitter put together a team of human editors to curate news and put trends in context. In addition, the top tweets that showed up under a trending topic would no longer just be algorithmically ranked, but include human curation as well. One year later, Twitter partnered with AP and Reuters to further bolster its efforts to provide human-provided context to trending topics.

Shortly after Musk bought the company in October 2022, though, the written context on trending topics disappeared. Musk had laid off Twitter’s human editors. As Reuters reported in November 2022, “Twitter’s curation team, which was responsible for ‘highlighting and contextualizing the greatest events and stories that unfold on Twitter,’ had been axed.”

X’s newly updated Explore page

Earlier this week, some X fan accounts like @XDaily shared screenshots of X’s upcoming Explore page relaunch. X’s Explore page includes the platform’s infamous trending topics list along with specific section breakouts like “News” and “Sports” to provide users with trending stories in each particular subject category.

Tweet may have been deleted

Musk’s updated Explore page looked poised to bring back the written context provided to trending topics and stories.

On April 4, X had began rolling out its newly updated Explore page. The update provides top user posts on each individual trending topic as well as an easy-to-digest summary of the subject matter above the user content. At the very top of the page, an X-created headline presents the information to readers much as if they were about to read an article on the topic. 

In addition, the update also gives Explore a new, prominent location on the platform. Explore’s trending stories now appear embedded directly in the right-hand sidebar of the main X homepage, where the trending topics list used to be located, which hundreds of millions of X’s daily users view everyday.

In a screenshot captured from popular Twitch user Hasan Piker’s stream, the fake AI-generated X headline can be seen trending on the platform’s homepage sidebar.
Credit: Hasan Piker’s Twitch

However, Musk has not brought back Twitter’s curation team, nor has he hired new human editors to write the context X now provides.

The context is written by X’s AI chatbot, Grok.

The danger of X depending on AI

We know that Grok, an AI feature heavily promoted by Musk, powers X’s contextualized summaries because X says so itself in the fine print on every Explore page.

Tweet may have been deleted

“Grok is an early feature and can make mistakes,” reads the tiny note from X underneath the written context provided on the Explore page. “Verify its outputs.”

Mashable observed X’s “Iran Strikes Tel Aviv with Heavy Missiles” page which spread fake news as it was trending on Thursday.

Based on our observations, it appears that the topic started trending because of a sudden uptick of blue checkmark accounts (users who pay a monthly subscription to X for Premium features including the verification badge) spamming the same copy-and-paste misinformation about Iran attacking Israel. The curated posts provided by X were full of these verified accounts spreading this fake news alongside an unverified video depicting explosions. 

A side-by-side comparison of two separate posts on the X Explore page showing fake news posts from blue check users that informed Grok.
Credit: Mashable screenshot

From there, it appears X’s algorithms noticed a potential story trend within these users’ posts, and an Explore story page was created. We can deduce from X’s own claims about its inner workings that Grok must have then created an official-looking written narrative, along with a catchy headline. It did all this based on select users sharing fake news, in an automated attempt to provide context for what the platform itself seemed to assume was a real story.

This wouldn’t be the first time Grok provided users with misinformation. Previous reporting on the early versions of X’s chatbot found that it often created fake news in private chats with the select few users who had access to it. However, this recent incident combined with the new Explore feature presents the first time X took Grok’s misinformation, packaged it as a real trending news story, and promoted it to its entire user base, ostensibly as context for a real event.

Under Musk, disinformation has skyrocketed on the platform known as X. Thanks to the new Explore product, the company promoted a falsehood, disseminated by paying bad actors who stand to make money in exchange for generating engagements.

One day after Grok generated this fake story that X then promoted via its Explore page, the company rolled Grok out to all Premium-subscribed X users so they could use the misinformation-creating AI chatbot as well.

Tweet may have been deleted