Lidar (light detection and ranging) has become a popular tool in vehicles to test self-driving systems of the future as well as implement new ways to use advanced driving assistance systems (ADAS).
But lidar is becoming a popular tool for other uses outside the vehicle, taking advantage of the technology’s ability to calculate the distance of objects via laser and generating 3D information about those objects.
Now, a team of researchers from the University of Maryland has used the laser-based navigation technology to hack a Roomba-type automated robot vacuum.
Researchers were able to use the lidar inside a vacuum robot and applied signal processing and deep learning to recover speech and identify television programs playing in the same room as the device. Researchers said the experiment shows that any device that uses lidar could potentially be manipulated for sound collection, despite not having a microphone.
"We welcome these devices into our homes, and we don't think anything about it," said Nirupam Roy, an assistant professor in the University of Maryland's Department of Computer Science. “But we have shown that even though these devices don't have microphones, we can repurpose the systems they use for navigation to spy on conversations and potentially reveal private information."
Lidar is used in robot vacuums to shine a laser beam in a room and sense the reflection of the laser as it bounces off objects nearby. The robot then uses the signals to map the room and avoid collisions as it moves through a house.
Potential privacy breaches could give advertisers access to information about the home or income level of a family or other related lifestyle data. Researchers said lidar in these robots could also pose potential security risks as sound recording devices inside homes.
While laser microphones have been used for decades to convert variations back into sound waves for espionage, lidar inside a robotic vacuum scans the environment with a laser and senses the light scattered back by objects that are irregular in shape and density. As such, researchers were unsure if a lidar system could be manipulated to function as a microphone and if the signal could reveal any meaningful sounds.
Hacking a vacuum
Researchers first hacked the vacuum to show if they could control the position of the laser beam and send the data to laptops through Wi-Fi without interfering with its navigation. They then conducted experiments with two sound sources: a human voice reciting numbers played over computer speakers and audio from television shows played through a TV sound bar. The team was able to capture laser signals sensed by the vacuum’s navigation system as it bounced off a variety of objects near the sound source.
Researchers then passed the signals through deep learning algorithms trained to either match human voices or identify musical sequences. The system, which the University of Maryland called Lidar Phone, identified and matched spoken numbers with 90% accuracy. It also identified TV shows from a minute’s worth of recording with 90% accuracy.
"This type of threat may be more important now than ever, when you consider that we are all ordering food over the phone and having meetings over the computer, and we are often speaking our credit card or bank information," Roy said. "But what is even more concerning for me is that it can reveal much more personal information. This kind of information can tell you about my living style, how many hours I'm working, other things that I am doing. And what we watch on TV can reveal our political orientations. That is crucial for someone who might want to manipulate the political elections or target very specific messages to me."
While the experiment was used on robot vacuums, the University of Maryland researchers said other devices that use lidar could be open to similar attacks or potentially even more devastating attacks. Just recently, Apple introduced its first 5G iPhone that included lidar and earlier this year the internet connected device giant released an iPad with lidar. Hacking these devices could allow hackers to get access to face recognition or passive infrared sensors used in motion detection.
