Facebook gets closer to letting you type with your mind

Jul 30, 2019, 1:33 PM

(CNN) — Elon Musk’s Neuralink isn’t the only company that wants to send your thoughts straight from your brain to a computer.

More than two years ago, Facebook revealed it was working on a project for typing words onto a computer right from your brain, without requiring invasive surgery to make it work.

The company has been working with several universities on the effort, including the University of California, San Francisco. Facebook helped pay for UCSF researchers to study whether electrodes placed in the brain could help us learn to “decode” speech from brainwaves in real-time.

As it turns out, this is possible: A study published Tuesday showed that researchers could instantly see — as text on a computer screen — a word or phrase that a participant was saying by looking at brain activity, as long as it was among a limited set of answers in response to predetermined questions. The study includes three epilepsy patients voluntarily implanted with electrodes.

Facebook is also footing the bill for a new, year-long study that UCSF is currently conducting where it will try to use brain activity to help a person who can’t speak communicate. The social network hopes the efforts could help reveal which brain signals are key for that non-invasive wearable that it’s planning for in the years ahead.

“We expect that to take upwards of 10 years,” Mark Chevillet, a research director at Facebook Reality Labs who runs its brain-computer interface group, told CNN Business of the overall project. “This is a long-term research program.”

An idea years in the making

At Facebook’s 2017 developer conference, F8, the company painted a fantastical picture of a mysterious, noninvasive device that would pick up on your brain signals and one day enable you to type 100 words per minute.

Such a gadget would be a far cry from the brain-computer interfaces scientists have been working on for decades. They still tend to be stuck in labs because they are pricey, have to be implanted under a user’s skull, and need to be connected to a computer to perform even the simplest tasks.

Not much has been heard about Facebook’s research since. Six months after showing it off on stage, Regina Dugan, the leader of Facebook’s secretive hardware division, Building 8, left the company (Building 8’s research efforts have since been added to Facebook Reality Labs, a which includes a range of virtual- and augmented-reality research). More recently, Facebook CEO Mark Zuckerberg mentioned it during a talk at Harvard in March, saying he’s excited about “reshaping our computing platforms to be fundamentally more about people and how we process the world.”

Now, however, some of Facebook’s efforts are coming to light with news of the ongoing research projects.

Electrodes on their brains

Participants in UCSF’s study are people with epilepsy who had a small patch of electrodes implanted on their brains in hopes of figuring out where their epileptic seizures originated; they volunteered to help with the Facebook-related study while in the hospital.

For the work related to Facebook, researchers had participants listen to questions while tracking their brain activity. Machine-learning algorithms eventually determined how to spot when participants were answering a question and which one of 24 answers they were choosing.

Translating what happens inside your brain into words is hard and doing it immediately is even harder. The speech decoding algorithms the researchers used were accurate up to 61% of the time at figuring out which of two dozen standard responses a participant spoke, right after the person was done talking.

This may not sound all that impressive, but Edward Chang, a neurosurgeon and professor of neurosurgery at UCSF who coauthored the study released this week believes it’s a “really important result” that could help people who have lost the ability to speak.

Facebook’s Chevillet, who previously worked at Johns Hopkins University as an adjunct professor of neuroscience, said the UCSF work is an “important yet somewhat expected milestone,” as the interpretation of brain signals tends to be done offline rather than in real time. Facebook, he said, has no interest in making any sort of medical device; what it wants is to understand the neural signals needed to create a silent speech interface.

A year-long study

Chang called the work in the paper published this week a “proof of concept.” Another Facebook-sponsored brain-computer-interface project he recently began in his lab is much lengthier: Chang will spend a year working with a single patient (a male who can no longer speak), tracking his brain activity with the same kind of electrode array used on the epilepsy patients, in hopes of restoring some of his communication abilities.

“We’ve got a tall order ahead of us to figure out how to make that work,” Chang said.

If it does, it could one day help a range of people, from those who have lost the ability to speak due to various brain-related injuries, to people who simply want to control a computer or send a message with their mind.

Meanwhile, at Facebook…

Chevillet said his group at Facebok is continuing its work on finding noninvasive ways to figure out what’s happening within the brain, too. It’s investigating how light may be able to indirectly track brain activity, specifically by using near-infrared light to measure oxygen saturation levels in your brain.

Though any sort of think-to-type device you might be able to buy is still far in the future, Chevillet can already imagine how he thinks it should look: a pair of glasses that uses augmented reality and includes a brain-based method for doing everything from sending a text message to adjusting the volume of a song to simply performing the equivalent of a mouse click.

“The use cases we envision are certainly for everybody,” he said.

Correction: The original version of this story misstated some details about the UCSF study.

™ & © 2019 Cable News Network, Inc., a Time Warner Company. All rights reserved.

Today’s Top Stories


FILE - This Friday, Feb. 16, 2018, file photo shows application icons from left, Facebook, Facebook...
Curt Gresseth

Expert gives tips on conquering your downward social media scroll

Are you addicted to the social-media scroll? An expert weighs in on what you can do to slow the scroll.
5 days ago
FILE - A Tesla owner charges his vehicle at a charging station in Topeka, Kan., Monday, April 5, 20...

US report: nearly 400 crashes of automated tech vehicles

However, the National Highway Traffic Safety Administration cautioned against using the numbers to compare automakers,
19 days ago
Iran digital intrusion...
Curt Gresseth

Don’t get hacked. Protect your online data by being cyber-ready.

A cyber-security says defend yourself against hackers by being cyber-ready.
2 months ago
The gold star on your license will be needed if you want to fly in the next year....
Dan Bammes

Your driver’s license needs this if you want to fly in the next year

The gold star on your driver's license will be needed for anyone 18 and older. Most people have been getting them as they've renewed their licenses over the past four or so years.
2 months ago
FILE - The Twitter splash page is seen on a digital device, Monday, April 25, 2022, in San Diego. E...
KELVIN CHAN AP Business Writer

Musk’s Twitter ambitions to collide with Europe’s tech rules

Europe's Digital Services Act requires big tech companies to police their platforms more strictly or face billions in fines.
2 months ago
secondary water davis weber...
Chandler Holt

WSU achieves water conservation goals four years early

Over 5 years ago, Weber State University set a water conservation goal to reduce campus water usage by 30% by 2025, a goal they achieved in 2021.
3 months ago

Sponsored Articles

Tax Harassment...
Jordan Wilcox

The best strategies for dealing with IRS tax harassment | You have options!

Learn how to deal with IRS tax harassment. This guide will teach you how to stop IRS phone calls and letters, and how to handle an IRS audit.
spend a day at Bear Lake...
Bear Lake Convention and Visitors Bureau

You’ll love spending the day at Bear Lake | How to spend a day at Bear Lake

Bear Lake is a place that needs to be experienced. Spend a day at Bear Lake.
Curb Appeal...
Price's Guaranteed Doors

How to have the best of both worlds for your house | Home security and curb appeal

Protect your home and improve its curb appeal with the latest security solutions like beautiful garage doors and increased security systems.
Prescription opioids can be disposed of during National Prescription Take Back Day...
Know Your Script

Prescription opioid misuse | How to protect your family from the opioid epidemic

Studies have shown that prescription opioid misuse has increased since COVID-19. So what do you need to know about these opioids?
national heart month...
Intermountain Healthcare

National Heart Month: 5 Lifestyle Changes to Make Today to Keep You Heart Healthy

Heart disease is the leading cause of death for both men and women. One person dies every 36 seconds in the United States from cardiovascular disease
Joseph Smith Memorial Building...
Temple Square

The Joseph Smith Memorial Building is an icon of Salt Lake City | Why hosting an event at this beautiful location will make you a hero this year

Here's why hosting an event at the iconic Joseph Smith Memorial Building in downtown Salt Lake City will make you a hero this year.
Facebook gets closer to letting you type with your mind