Audio and Video

Teaching smart hearing aids to understand speech

09 September 2022

A University of Glasgow-led team has put forth a study on how they set out to harness cutting-edge sensing technology to read lips. New “smart” hearing aids would combine conventional audio amplification with a second device to collect additional data for improved performance.

An international team of engineers and computing scientists developed the technology, which pairs radio-frequency (RF) sensing with artificial intelligence for the first time, along with traditional hearing aid technology, to identify lip movements.

Previous technological attempts used cameras to read lips, which raised obvious privacy issues and did not work with masks. Source: Peakstock/AdobePrevious technological attempts used cameras to read lips, which raised obvious privacy issues and did not work with masks. Source: Peakstock/Adobe

To develop the system, the researchers asked volunteers to repeat the five vowel sounds (A, E, I, O and U) first while unmasked and then while wearing a surgical mask. As the volunteers repeated the vowel sounds, a dedicated radar sensor and a Wi-Fi transmitter scanned their faces using RF signals. Researchers also scanned their faces while their lips remained still.

It could also help tackle the “cocktail party effect”, a common shortcoming of traditional hearing aids as the broad spectrum of amplification can make it difficult for users to focus on specific sounds, like conversation with one specific person in a crowd.

Previous similar attempts used cameras to read lips, which raised obvious privacy issues and did not work with masks. This method preserves privacy by collecting only RF data, with no accompanying video footage.

“Around five percent of the world’s population – about 430 million people – have some kind of hearing impairment," says Dr. Qammer Abbasi of the University of Glasgow’s James Watt School of Engineering in a university release. Dr. Abbasi is a co-researcher in the study published in Nature Communications.

“Hearing aids have provided transformative benefits for many hearing-impaired people. A new generation of technology which collects a wide spectrum of data to augment and enhance the amplification of sound could be another major step in improving hearing-impaired people’s quality of life.

“With this research, we have shown that radio-frequency signals can be used to accurately read vowel sounds on people’s lips, even when their mouths are covered. While the results of lip-reading with radar signals are slightly more accurate, the Wi-Fi signals also demonstrated impressive accuracy,” Dr. Abbasi continues.



Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Advertisement
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Advertisement
Find Free Electronics Datasheets
Advertisement