Facebook wants its augmented reality glasses to read your mind

It turns out Facebook’s research into brain-reading computers is still very much alive, and the work could have significant implications for another one of the company’s ambitious projects: augmented reality glasses. 

On Tuesday, Facebook gave its first significant update on its brain-computer interface research since it first introduced the project onstage at F8 in 2017. The ultimate goal of the work, as Facebook has described it, is to create a system that can “decode silent speech” without the need for implanting electrodes into the brain. 

Researchers from the University of California, San Francisco, who have been collaborating with Facebook, say they have taken an important step toward that goal in a new paper published in Nature. The researchers, who were working with people already having brain surgery for epilepsy, created an algorithm that was able to “decode a small set of full, spoken words and phrases from brain activity in real time.”

Facebook notes that working with brain surgery patients is far from the “non-invasive” approach they’re hoping for, though. To solve this, Facebook’s Reality Labs, home to the company’s AR and VR research, is trying other methods. One involves trying to detect brain activity by monitoring oxygen levels in the brain with “a portable, wearable device made from consumer-grade parts.”

“It’s currently bulky, slow, and unreliable,” according to Facebook, but it could still one day make its way into the company’s virtuality reality headsets or its planned augmented reality glasses.

“While measuring oxygenation may never allow us to decode imagined sentences, being able to recognize even a handful of imagined commands, like ‘home,’ ‘select,’ and ‘delete,’ would provide entirely new ways of interacting with today’s VR systems — and tomorrow’s AR glasses,” Facebook writes in a statement.

“Thanks to the commercialization of optical technologies for smartphones and LiDAR, we think we can create small, convenient BCI devices that will let us measure neural signals closer to those we currently record with implanted electrodes — and maybe even decode silent speech one day.”

This adds an interesting new wrinkle to Facebook’s planned AR glasses, which we still know relatively little about. Oculus chief scientist Michael Abrash said in 2017 the glasses were at least five, but possibly 10, years away. Since then, much of what we know has come from sporadic patent filings. If Facebook does intend for the glasses to have mind-reading abilities, then the 10-year timeline seems more likely. 

Of course, the prospect of Facebook, a company which was just hit with a record-breaking $5 billion fine for violating its users’ privacy, being able to literally detect the thoughts in your head might not be particularly appealing. For what it’s worth, Facebook’s Reality Labs scientists say they are already thinking about the ethical implications of the tech, though they don’t give any specifics.

“It’s a tantalizing vision, but one that will require an enterprising spirit, hefty amounts of determination, and an open mind,” Facebook wrote.