PRESS-NEWS.org - Press Release Distribution
PRESS RELEASES DISTRIBUTION

Brain-to-voice neuroprosthesis restores naturalistic speech

AI-based model streams intelligible speech from the brain in real time

2025-03-31
(Press-News.org) Marking a breakthrough in the field of brain-computer interfaces (BCIs), a team of researchers from UC Berkeley and UC San Francisco has unlocked a way to restore naturalistic speech for people with severe paralysis.

This work solves the long-standing challenge of latency in speech neuroprostheses, the time lag between when a subject attempts to speak and when sound is produced. Using recent advances in artificial intelligence-based modeling, the researchers developed a streaming method that synthesizes brain signals into audible speech in near-real time.

As reported in Nature Neuroscience, this technology represents a critical step toward enabling communication for people who have lost the ability to speak. The study is supported by the National Institute on Deafness and Other Communication Disorders (NIDCD) of the National Institutes of Health.

“Our streaming approach brings the same rapid speech decoding capacity of devices like Alexa and Siri to neuroprostheses,” said Gopala Anumanchipalli, Robert E. and Beverly A. Brooks Assistant Professor of Electrical Engineering and Computer Sciences at UC Berkeley and co-principal investigator of the study. “Using a similar type of algorithm, we found that we could decode neural data and, for the first time, enable near-synchronous voice streaming. The result is more naturalistic, fluent speech synthesis.”

“This new technology has tremendous potential for improving quality of life for people living with severe paralysis affecting speech,” said neurosurgeon Edward Chang, senior co-principal investigator of the study. Chang leads the clinical trial at UCSF that aims to develop speech neuroprosthesis technology using high-density electrode arrays that record neural activity directly from the brain surface. “It is exciting that the latest AI advances are greatly accelerating BCIs for practical real-world use in the near future.”

The researchers also showed that their approach can work well with a variety of other brain sensing interfaces, including microelectrode arrays (MEAs) in which electrodes penetrate the brain’s surface, or non-invasive recordings (sEMG) that use sensors on the face to measure muscle activity.

“By demonstrating accurate brain-to-voice synthesis on other silent-speech datasets, we showed that this technique is not limited to one specific type of device,” said Kaylo Littlejohn, Ph.D. student at UC Berkeley's Department of Electrical Engineering and Computer Sciences and co-lead author of the study. “The same algorithm can be used across different modalities provided a good signal is there.”

Decoding neural data into speech

According to study co-lead author Cheol Jun Cho, UC Berkeley Ph.D. student in electrical engineering and computer sciences, the neuroprosthesis works by sampling neural data from the motor cortex, the part of the brain that controls speech production, then uses AI to decode brain function into speech.

“We are essentially intercepting signals where the thought is translated into articulation and in the middle of that motor control,” he said. “So what we’re decoding is after a thought has happened, after we’ve decided what to say, after we’ve decided what words to use and how to move our vocal-tract muscles.”

To collect the data needed to train their algorithm, the researchers first had Ann, their subject, look at a prompt on the screen — like the phrase: “Hey, how are you?” — and then silently attempt to speak that sentence.

“This gave us a mapping between the chunked windows of neural activity that she generates and the target sentence that she’s trying to say, without her needing to vocalize at any point,” said Littlejohn.

Because Ann does not have any residual vocalization, the researchers did not have target audio, or output, to which they could map the neural data, the input. They solved this challenge by using AI to fill in the missing details.

“We used a pretrained text-to-speech model to generate audio and simulate a target,” said Cho. “And we also used Ann’s pre-injury voice, so when we decode the output, it sounds more like her.”

Streaming speech in near real time

In their previous BCI study, the researchers had a long latency for decoding, about an 8-second delay for a single sentence. With the new streaming approach, audible output can be generated in near-real time, as the subject is attempting to speak.

To measure latency, the researchers employed speech detection methods, which allowed them to identify the brain signals indicating the start of a speech attempt.

“We can see relative to that intent signal, within 1 second, we are getting the first sound out,” said Anumanchipalli. “And the device can continuously decode speech, so Ann can keep speaking without interruption.”

This greater speed did not come at the cost of precision. The faster interface delivered the same high level of decoding accuracy as their previous, non-streaming approach.

“That’s promising to see,” said Littlejohn. “Previously, it was not known if intelligible speech could be streamed from the brain in real time.”

Anumanchipalli added that researchers don’t always know whether large-scale AI systems are learning and adapting, or simply pattern-matching and repeating parts of the training data. So the researchers also tested the real-time model’s ability to synthesize words that were not part of the training dataset vocabulary — in this case, 26 rare words taken from the NATO phonetic alphabet, such as “Alpha,” “Bravo,” “Charlie” and so on.

“We wanted to see if we could generalize to the unseen words and really decode Ann’s patterns of speaking,” he said. “We found that our model does this well, which shows that it is indeed learning the building blocks of sound or voice.”

Ann, who also participated in the 2023 study, shared with researchers how her experience with the new streaming synthesis approach compared to the earlier study’s text-to-speech decoding method.

“She conveyed that streaming synthesis was a more volitionally controlled modality,” said Anumanchipalli. “Hearing her own voice in near-real time increased her sense of embodiment.” 

Future directions

This latest work brings researchers a step closer to achieving naturalistic speech with BCI devices, while laying the groundwork for future advances.

“This proof-of-concept framework is quite a breakthrough,” said Cho. “We are optimistic that we can now make advances at every level. On the engineering side, for example, we will continue to push the algorithm to see how we can generate speech better and faster.”

The researchers also remain focused on building expressivity into the output voice to reflect the changes in tone, pitch or loudness that occur during speech, such as when someone is excited.

“That’s ongoing work, to try to see how well we can actually decode these paralinguistic features from brain activity,” said Littlejohn. “This is a longstanding problem even in classical audio synthesis fields and would bridge the gap to full and complete naturalism.”

In addition to the NIDCD, support for this research was provided by the Japan Science and Technology Agency’s Moonshot Research and Development Program, the Joan and Sandy Weill Foundation, Susan and Bill Oberndorf, Ron Conway, Graham and Christina Spencer, the William K. Bowes, Jr. Foundation, the Rose Hills Innovator and UC Noyce Investigator programs, and the National Science Foundation.

END


ELSE PRESS RELEASES FROM THIS DATE:

NCSA research offers new diagnostic tool for anxiety, major depressive disorder

2025-03-31
Scientists at the National Center for Supercomputing Applications and the University of Illinois College of Medicine Peoria (UICOMP) were authors of a research paper published in the Journal of Acoustical Society of America Express Letters that demonstrates improved, automated screening methods for anxiety and major depressive disorders. In the project titled, “Automated acoustic voice screening techniques for comorbid depression and anxiety disorders,” Mary Pietrowicz, along with colleagues from the University of Illinois Urbana-Champaign and UICOMP, explored how machine ...

Single protein mimics retinoic acid therapy to help leukemia cells mature

Single protein mimics retinoic acid therapy to help leukemia cells mature
2025-03-31
“This study reveals that the expression of the Src-family kinase (SFK), FGR, alone can induce cell differentiation similar to RA.” BUFFALO, NY – March 31, 2025 – A new research paper was published in Oncotarget, Volume 16, on March 21, 2025, titled “FGR Src family kinase causes signaling and phenotypic shift mimicking retinoic acid-induced differentiation of leukemic cells.” A research team led by first author Noor Kazim and corresponding author Andrew Yen from Cornell University discovered that the FGR protein—traditionally ...

Research: Top scientists issue urgent warning on fossil fuels

2025-03-31
WASHINGTON— In a review published today in the peer-reviewed journal Oxford Open Climate Change, top scientists issued an urgent warning that fossil fuels and the fossil fuel industry are driving interlinked crises that threaten people, wildlife, and a livable future.  Today’s review synthesizes the extensive scientific evidence showing that fossil fuels and the fossil fuel industry are fueling not only the climate crisis but also public health harms, environmental injustice, biodiversity loss, and the plastics ...

Thinner Arctic sea ice may affect the AMOC

Thinner Arctic sea ice may affect the AMOC
2025-03-31
One of the ocean currents in the Arctic Ocean is at risk of disappearing this century because of climate change, according to a new joint study from the University of Gothenburg and the German Alfred Wegener Institute. As a result, the North Atlantic could be flooded with freshwater which would weaken the global ocean circulation.   The weakening of the AMOC, Atlantic Meridional Overturning Circulation, is a hot topic among the world's climate scientists. However, it is unclear what the consequences will be when ...

How dreams, novelty, and emotions can shape memories: lessons from smartphone studies

2025-03-31
BOSTON - March 31, 2025 - A memory is not a straight line from one point to another, even if we sometimes think of them like linear stories. This key insight that cognitive neuroscientists have known for many years is now guiding a new type of research—to explore not only how memories evolve over time but also how they can be strengthened or changed. Assisting researchers with this new exploration is a powerful tool: smartphones. “Smartphones are an incredible tool for understanding patterns of feelings, behavior, ...

Leveraging Preexisting Cardiovascular Data to Improve the Detection and Treatment of Hypertension

2025-03-31
About The Study: In the NOTIFY-LVH randomized clinical trial, a centralized population health coordinator–led notification and clinical support pathway for individuals with left ventricular hypertrophy on prior echocardiograms increased the initial treatment of hypertension. This work highlights the potential benefit of leveraging preexisting but potentially underutilized cardiovascular data to improve health care delivery through mechanisms augmenting the traditional ambulatory care system. Corresponding Author: To contact the corresponding author, Jason H. Wasfy, MD, MPhil, email jwasfy@mgh.harvard.edu. To access the embargoed study: Visit ...

Study highlights barriers to including non-English speakers in research

Study highlights barriers to including non-English speakers in research
2025-03-31
Health researchers omit including speakers of languages other than English (LOE) in their studies due to lack of training, challenges securing interpreter services, budget constraints and other barriers, according to a new JAMA Network Open study. “Non-English-speaking individuals are being excluded from research, and we wanted to understand why,” said senior author Maya Ragavan, M.D., M.P.H., M.S., associate professor of pediatrics at the University of Pittsburgh. “Ensuring that everyone can participate in research, regardless of the languages that they speak, helps ensure that the findings are representative of the entire community, ...

Queen Mary launches new regenerative medicine spinout to repair joint damage and prevent arthritis

2025-03-31
The method is substantially cheaper and easier to administer than even the best of today’s approaches – making it significantly more viable for healthcare providers.   The invention paves the way for a future restorative treatment for osteoarthritis.  Scientists at Queen Mary University of London have invented a new way to repair osteochondral defects – a common form of joint damage often caused by sports injuries or trauma – and to prevent progression to osteoarthritis.   The treatment uses Agrin, the protein which builds and maintains the link between our motor neurons and muscle fibres. A small soluble polypeptide ...

Make these scientific sessions part of your media coverage

2025-03-31
New York, NY – March 31, 2025 – Register now for a complimentary media pass for the ATS 2025 International Conference in San Francisco. We will have breaking news from the leading international conference for pulmonary, critical care and sleep medicine. See what scientific sessions we have planned: Allergy/Immunology  Critical Care  Cellular/Molecular Biology Bronchiectasis and COPD  Health Equity/Health Disparities  ILD  Pediatrics  Sleep All registered media will receive a press release one week prior to the conference with ...

Common anticancer drugs may offer new hope to PTEN Hamartoma Tumour Syndrome patients

Common anticancer drugs may offer new hope to PTEN Hamartoma Tumour Syndrome patients
2025-03-31
During development, cells grow, expand, and migrate to generate tissues and organs in a highly controlled manner. Many intracellular pathways – series of signalling cascades within a cell – regulate these actions to avoid non-programmed growth that could lead to malformations or cancer. One of these pathways is the PTEN / PI3K axis, a complex series of perfectly balanced chemical reactions. Mutations in the PTEN gene usually result in the overactivation of PI3K and the imbalance of the system. This may trigger the onset of different types ...

LAST 30 PRESS RELEASES:

Inability to pay for healthcare reaches record high in U.S.

Science ‘storytelling’ urgently needed amid climate and biodiversity crisis

KAIST Develops Retinal Therapy to Restore Lost Vision​

Adipocyte-hepatocyte signaling mechanism uncovered in endoplasmic reticulum stress response

Mammals were adapting from life in the trees to living on the ground before dinosaur-killing asteroid

Low LDL cholesterol levels linked to reduced risk of dementia

Thickening of the eye’s retina associated with greater risk and severity of postoperative delirium in older patients

Almost one in ten people surveyed report having been harmed by the NHS in the last three years

Enhancing light control with complex frequency excitations

New research finds novel drug target for acute myeloid leukemia, bringing hope for cancer patients

New insight into factors associated with a common disease among dogs and humans

Illuminating single atoms for sustainable propylene production

New study finds Rocky Mountain snow contamination

Study examines lactation in critically ill patients

UVA Engineering Dean Jennifer West earns AIMBE’s 2025 Pierre Galletti Award

Doubling down on metasurfaces

New Cedars-Sinai study shows how specialized diet can improve gut disorders

Making moves and hitting the breaks: Owl journeys surprise researchers in western Montana

PKU Scientists simulate the origin and evolution of the North Atlantic Oscillation

ICRAFT breakthrough: Unlocking A20’s dual role in cancer immunotherapy

How VR technology is changing the game for Alzheimer’s disease

A borrowed bacterial gene allowed some marine diatoms to live on a seaweed diet

Balance between two competing nerve proteins deters symptoms of autism in mice

Use of antifungals in agriculture may increase resistance in an infectious yeast

Awareness grows of cancer risk from alcohol consumption, survey finds

The experts that can outsmart optical illusions

Pregnancy may reduce long COVID risk

Scientists uncover novel immune mechanism in wheat tandem kinase

Three University of Virginia Engineering faculty elected as AAAS Fellows

Unintentional drug overdoses take a toll across the U.S. unequally, study finds

[Press-News.org] Brain-to-voice neuroprosthesis restores naturalistic speech
AI-based model streams intelligible speech from the brain in real time