Researchers have developed software that makes it possible to use low-cost thermal cameras attached to mobile phones to track breathing. Mobile thermal imaging could be used to monitor breathing problems with elderly people who live alone, people who may have sleep apnea or babies that are at risk for sudden infant death syndrome (SIDS).
The researchers say their new software, when combined with a low-cost thermal camera, performed well when analyzing breathing rate during tests that simulate real-world movement and temperature changes.
"As thermal cameras continue to get smaller and less expensive, we expect that phones, computers and augmented reality devices will one day incorporate thermal cameras that can be used for various applications," said Nadia Bianchi-Berthouze from University College London, (UK) and leader of the research team. "By using low-cost thermal cameras, our work is a first step toward bringing thermal imaging into people's everyday lives. This approach can be used in places other sensors might not work or would cause concern."
Example shots of conversion from thermal images to thermal-gradient magnitude maps: the proposed method can help to preserve the morphology of the nostril region during motion (Zoomed-in-areas are manually rotated for the visual representation) (Youngjun Cho)
Along with detecting breathing problems, the approach could allow cameras on a computer to detect subtle breathing irregularities that are associated with pain or stress and send prompts to help the user relax or regulate their breathing. Traditional video cameras can be used to track breathing, but they don’t work in low-light situations and can also cause privacy concerns when it is used for monitoring in places like nursing homes.
"Thermal cameras can detect breathing at night and during the day without requiring the person to wear any type of sensor," said Youngjun Cho, first author of the paper. "Compared to a traditional video camera, a thermal camera is more private because it is more difficult to identify the person."
Thermal cameras use infrared wavelengths to reveal the temperature of an object or scene. They have been used in a variety of monitoring applications for a long time. The price and size have dropped enough recently that makes them practical for personal use and small thermal cameras that connect to mobile phones are currently available for about $200.
"Large, expensive thermal imaging systems have been used to measure breathing by monitoring temperature changes inside the nostrils under controlled settings," said Cho. "We wanted to use the new portable systems to do the same thing by creating a smart-phone based respiratory tracking method that could be used in almost any environment or activity. However, we found that in real-world situations this type of mobile thermal imaging was affected by changes in air temperature and body movement."
In order to solve these problems, the researchers developed algorithms that can be used with any thermal camera to compensate for ambient temperature changes and track nostrils while a person is moving. The new algorithms improve the way breathing signals are processed. Instead of averaging temperature readings from 2D pixels around the nostrils, Cho developed a way to treat the area as a 3D surface to create a refined measurement of temperature in the nostrils.
Researchers used the mobile thermal imaging approach to measure the breaking of volunteers in a scenario that involved breathing exercises with a change of ambient temperature in an unconstrained test. In this test, volunteers walked around in the inside and outside of the building. During the walking tests, the thermal camera was placed between 20 and 30 centimeters from a person’s face using a rig that attached the camera to the hat. A cord connected the camera with a mobile phone. It is possible to hold the smartphone with an imaging camera about 50 centimeters from the face to measure breathing.
"For all three types of studies, the algorithms showed significantly better performance in tracking the nostril area than other state-of-the-art methods," said Cho. "In terms of estimating the breathing rate, the tests outside the laboratory showed the best results when compared with the latest algorithms. Although the results were comparable to the traditional breathing belt sensor, for mobile situations our approach seems to be more stable because the belt tends to get loose."
This new approach is a more stable than standard chest belt respiratory sensors -- the method could potentially be used to optimize an athlete’s performance by providing reliable and accurate feedback on breathing patterns during activities, like exercise or practice.
The researchers inferred a person’s mental load or stress through automatic breathing analysis. They used thermal imaging software to track the breaking of people who were free to move around while performing tasks. The results aligned with findings from other studies that used more sophisticated equipment. This indicated the portable thermal-camera based approach that could be useful for apps that help people relax.
"By using mobile thermal imaging to monitor only breathing, we obtained results very comparable to what other studies had found," said Bianchi-Berthouze. "However, those studies used complex, state-of-the-art techniques that involved multiple sensors monitoring not just breathing but also heart rate."
Current versions of this software doesn’t estimate breathing rate in real time, but researchers are working on incorporating real time capability.
A paper on this research was published in The Optical Society journal Biomedical Optics Express
