Home Appliances

Computer Scientists Use Music to Track Body Movements

17 August 2017

Computer scientists have been concerned that smartphones, tablets, smart TVs and other smart devices, if not properly secured, might be co-opted to steal data or invade user privacy.

Researchers at the University of Washington have proved that it is possible to transform a smart device into a surveillance tool that can collect information about the body position and movements of the user and other devices that are in the area. The researchers' approach involved remotely hijacking smart devices to play music embedded with a repeating pulse that tracks the user’s body position, movements and activities in the vicinity of the device and through the walls.

The team is from UW’s Paul G. Allen School of Computer Science and Engineering. They showed that it is possible to collect detailed data on personal activity through the CoverBand, software code they created to turn smart devices into sonar systems.

"To our knowledge, this is the first time anyone has demonstrated that it is possible to convert smart commodity devices--like smartphones and smart TVs--into active sonar systems using music," said senior author Shyam Gollakota, a UW associate professor of computer science and engineering. "And the physical information CovertBand can gather--even through walls--is sufficiently detailed for an attacker to know what the user is doing, as well as other people nearby."

The researchers tested CovertBand using a Samsung Galaxy S4 smartphone hooked up to a portable speaker. (Dennis Wise/University of Washington)The researchers tested CovertBand using a Samsung Galaxy S4 smartphone hooked up to a portable speaker. (Dennis Wise/University of Washington)

CovertBand uses the principles of active sonar to gather the information. Active sonar systems like those used on submarines determine the position of objects by sending out an acoustic pulse. The sound waves bounce off objects in their path and the reflected waves can be picked up by a receiver to determine the object’s position, distance from the receiver and shape.

CoverBand sends out a repeating pulse of sound waves in the 18-20 kHz range through the speaker of the smart device. Like the sonar of a submarine, these sound waves are reflected when they encounter objects in their path. CovertBand uses the device’s built-in microphones as a receiver to pick up the reflected sound waves. The smart device transmits this information to the attacker from a few feet away or from across the country or the globe.

"Most of today's smart devices including smart TVs, Google Home, Amazon Echo and smartphones come with built-in microphones and speaker systems--which lets us use them to play music, record video, and audio tracks, have phone conversations or participate in video conferencing," said co-lead author Rajalakshmi Nandakumar, a UW doctoral student in computer science and engineering. "But that also means that these devices have the basic components in place to make them vulnerable to attack in this manner."

"Other surveillance approaches require specialized hardware, from the 'classic' hidden camera to an ultrasound-like device that must be placed on the wall of a neighboring room," said co-lead author Alex Takakuwa, a UW doctoral student in computer science and engineering. "CovertBand shows for the first time that through-barrier surveillance is possible using no hardware beyond what smart devices already have."

The team tested CoverBand using a smartphone hooked up to a portable speaker or a standard flat-screen TV. CoverBand’s data could be used to decipher repetitive movements like arm-pumping, walking or pelvic tilts to a range of up to 6 meters from the smartphone. It has a positional error of only 8 to 18 centimeters. Researchers discovered that when used with a portable speaker, CovertBand’s pulses can transmit through thin, interior walls but the range drops 2 to 3 meters.

CovertBand can currently automatically identify and infer only repetitive motions. More detailed inferences require manual analyses of data or additional tools.

"Our initial goal was to demonstrate that it is possible to use passive acoustics to gather even basic -- but still highly sensitive -- information using CovertBand," said Gollakota. "But if you have enough data from CovertBand, you could run it through machine-learning algorithms to help classify more movements for faster identification."

The 18 to 20 kHz repeating pulses employed by CovertBand are on the low range of what many people can hear accurately, but children, younger adults and pets might be able to hear it very well. To increase the range of surveillance and work through walls, the team increased the volume of repeating pulses to make them audible. In order to mask the count, the researchers “covered” CovertBand’s compositions with repetitive and percussive beats. When the team played the CovertBand pulses beneath 20 popular songs, listeners could identify the “hacked” version 58 percent of the time.

"Since CovertBand enables through-the-wall surveillance, anyone can play music on their smart devices to track people through walls," said Takakuwa. "This is concerning because, if a neighbor is playing music, it could either be a benign act or an act of surveillance to determine if anyone is in the adjacent apartment, track their movements or infer their activities."

The researchers noted that a soundproof room would prevent attacks through walls. Emitting a jamming signal at 18 to 20 kHz frequency range would prevent hacked devices or attackers in the next room from gathering information. Currently, these are impractical defenses for most people. Soundproofed rooms don’t have windows, and jamming signals would have to be sent the minute an attack is detected. Another potential defense would be for users to deactivate the speakers or microphones on their smart devices. But this would go against industry trends for smart devices.

The team hopes that the knowledge they have discovered will help develop awareness of privacy dangers and prompt scientists to develop countermeasures that work for everyone.

This research will be reported at the Ubicomp 2017 conference in September.

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Find Free Electronics Datasheets