Facebook gets closer to letting you type with your mind

Jul 30, 2019, 1:33 PM

(CNN) — Elon Musk’s Neuralink isn’t the only company that wants to send your thoughts straight from your brain to a computer.

More than two years ago, Facebook revealed it was working on a project for typing words onto a computer right from your brain, without requiring invasive surgery to make it work.

The company has been working with several universities on the effort, including the University of California, San Francisco. Facebook helped pay for UCSF researchers to study whether electrodes placed in the brain could help us learn to “decode” speech from brainwaves in real-time.

As it turns out, this is possible: A study published Tuesday showed that researchers could instantly see — as text on a computer screen — a word or phrase that a participant was saying by looking at brain activity, as long as it was among a limited set of answers in response to predetermined questions. The study includes three epilepsy patients voluntarily implanted with electrodes.

Facebook is also footing the bill for a new, year-long study that UCSF is currently conducting where it will try to use brain activity to help a person who can’t speak communicate. The social network hopes the efforts could help reveal which brain signals are key for that non-invasive wearable that it’s planning for in the years ahead.

“We expect that to take upwards of 10 years,” Mark Chevillet, a research director at Facebook Reality Labs who runs its brain-computer interface group, told CNN Business of the overall project. “This is a long-term research program.”

An idea years in the making

At Facebook’s 2017 developer conference, F8, the company painted a fantastical picture of a mysterious, noninvasive device that would pick up on your brain signals and one day enable you to type 100 words per minute.

Such a gadget would be a far cry from the brain-computer interfaces scientists have been working on for decades. They still tend to be stuck in labs because they are pricey, have to be implanted under a user’s skull, and need to be connected to a computer to perform even the simplest tasks.

Not much has been heard about Facebook’s research since. Six months after showing it off on stage, Regina Dugan, the leader of Facebook’s secretive hardware division, Building 8, left the company (Building 8’s research efforts have since been added to Facebook Reality Labs, a which includes a range of virtual- and augmented-reality research). More recently, Facebook CEO Mark Zuckerberg mentioned it during a talk at Harvard in March, saying he’s excited about “reshaping our computing platforms to be fundamentally more about people and how we process the world.”

Now, however, some of Facebook’s efforts are coming to light with news of the ongoing research projects.

Electrodes on their brains

Participants in UCSF’s study are people with epilepsy who had a small patch of electrodes implanted on their brains in hopes of figuring out where their epileptic seizures originated; they volunteered to help with the Facebook-related study while in the hospital.

For the work related to Facebook, researchers had participants listen to questions while tracking their brain activity. Machine-learning algorithms eventually determined how to spot when participants were answering a question and which one of 24 answers they were choosing.

Translating what happens inside your brain into words is hard and doing it immediately is even harder. The speech decoding algorithms the researchers used were accurate up to 61% of the time at figuring out which of two dozen standard responses a participant spoke, right after the person was done talking.

This may not sound all that impressive, but Edward Chang, a neurosurgeon and professor of neurosurgery at UCSF who coauthored the study released this week believes it’s a “really important result” that could help people who have lost the ability to speak.

Facebook’s Chevillet, who previously worked at Johns Hopkins University as an adjunct professor of neuroscience, said the UCSF work is an “important yet somewhat expected milestone,” as the interpretation of brain signals tends to be done offline rather than in real time. Facebook, he said, has no interest in making any sort of medical device; what it wants is to understand the neural signals needed to create a silent speech interface.

A year-long study

Chang called the work in the paper published this week a “proof of concept.” Another Facebook-sponsored brain-computer-interface project he recently began in his lab is much lengthier: Chang will spend a year working with a single patient (a male who can no longer speak), tracking his brain activity with the same kind of electrode array used on the epilepsy patients, in hopes of restoring some of his communication abilities.

“We’ve got a tall order ahead of us to figure out how to make that work,” Chang said.

If it does, it could one day help a range of people, from those who have lost the ability to speak due to various brain-related injuries, to people who simply want to control a computer or send a message with their mind.

Meanwhile, at Facebook…

Chevillet said his group at Facebok is continuing its work on finding noninvasive ways to figure out what’s happening within the brain, too. It’s investigating how light may be able to indirectly track brain activity, specifically by using near-infrared light to measure oxygen saturation levels in your brain.

Though any sort of think-to-type device you might be able to buy is still far in the future, Chevillet can already imagine how he thinks it should look: a pair of glasses that uses augmented reality and includes a brain-based method for doing everything from sending a text message to adjusting the volume of a song to simply performing the equivalent of a mouse click.

“The use cases we envision are certainly for everybody,” he said.

Correction: The original version of this story misstated some details about the UCSF study.

™ & © 2019 Cable News Network, Inc., a Time Warner Company. All rights reserved.

Today’s Top Stories


In this image provided by NASA, the Earth and its moon are seen from NASA's Orion spacecraft on Mon...
Jackie Wattles, CNN

NASA’s Orion reaches record-breaking distance from Earth

NASA confirmed that Orion had reached the midpoint of its uncrewed mission around the moon — about 270,000 miles (434,523 kilometers) from Earth.
1 day ago
Former US President Donald Trump's Twitter account has been reinstated on the platform....
Clare Duffy and Paul LeBlanc, CNN

Elon Musk restores Donald Trump’s Twitter account

Former US President Donald Trump's Twitter account has been reinstated on the platform.
11 days ago
With employees with the company, the future of Twitter is in uncertain. Photo credit: Justin Sulliv...
Oliver Darcy, CNN Business

Inside Twitter as ‘mass exodus’ of staffers throws platform’s future into uncertainty

With employees leaving the company, the future of Twitter is uncertain.
12 days ago
NASA's Space Launch System (SLS) rocket with the Orion spacecraft aboard is seen on Nov. 12 at NASA...
Jackie Wattles, CNN

Historic moon mission troubleshoots fuel leak issue hours before launch

The Artemis I mission to the moon could finally take place this week.
15 days ago
FILE - A Tesla owner charges his vehicle at a charging station in Topeka, Kan., Monday, April 5, 20...
Peter Valdes-Dapena, CNN Business

Tesla officially makes its charging standard available to other companies

Tesla has invited other automakers to build cars with charging ports that can work with Tesla's charging format and for other charging companies to add Tesla-style plugs to their chargers.
18 days ago
Former US President Donald Trump's Twitter account has been reinstated on the platform....
BARBARA ORTUTAY AP Technology Writer

Twitter Blue signups unavailable after raft of fake accounts

Blue checkmark "verification" was unavailable for purchase Friday after the social media platform was flooded by impostor accounts approved by Twitter.
19 days ago

Sponsored Articles

Happy joyful smiling casual satisfied woman learning and communicates in sign language online using...

The best tools for Deaf and hard-of-hearing workplace success

Here are some of the best resources to make your workplace work better for Deaf and hard-of-hearing employees.
Team supporters celebrating at a tailgate party...

8 Delicious Tailgate Foods That Require Zero Prep Work

In a hurry? These 8 tailgate foods take zero prep work, so you can fuel up and get back to what matters most: getting hyped for your favorite
christmas decorations candles in glass jars with fir on a old wooden table...
Western Nut Company

12 Mason Jar Gift Ideas for the 12 Days of Christmas [with recipes!]

There are so many clever mason jar gift ideas to give something thoughtful to your neighbors or friends. Read our 12 ideas to make your own!
wide shot of Bear Lake with a person on a stand up paddle board...

Pack your bags! Extended stays at Bear Lake await you

Work from here! Read our tips to prepare for your extended stay, whether at Bear Lake or somewhere else nearby.
young boy with hearing aid...

Accommodations for students who are deaf and hard of hearing

These different types of accommodations for students who are deaf and hard of hearing can help them succeed in school.
Young woman receiving laser treatment...
Form Derm Spa

How facial plastic surgery and skincare are joining forces

Facial plastic surgery is not only about looking good but about feeling good too. The medical team at Form Spa are trained to help you reach your aesthetic outcomes through surgery and through skincare and dermatology, too.
Facebook gets closer to letting you type with your mind