Audio and Video

Smartwatch Can Detect Taps, Flicks and Scratches on the Wearers Body

17 October 2016

New smartwatch technology created by researchers from Carnegie Mellon University enables smartwatches to detect taps, scratches and flicks against a wearer's body, opening the door to new forms of interaction with wearable devices.

The new technology, which the team calls ViBand, was made by possible by a simple software upgrade that repurposes the device's existing accelerometer.

Carnegie Mellon University researchers show off their new technology using a repurposed sensor. (Image Credit: CMU Future Interfaces Group)Carnegie Mellon University researchers show off their new technology using a repurposed sensor. (Image Credit: CMU Future Interfaces Group)

The discovery paves the way for new applications that use common gestures to control smartwatches and even other objects that are connected to the IoT. Since the smartwatch monitors vibrations that occur when people hold objects or use tools, it would also be able to differentiate between objects and activities.

According to the researchers, the new feature could even help tune a guitar, with the smartwatch displaying the note transmitted as the guitarist plucks and adjusts each string.

"It's as if you're using your hand as a detection device," said Gierad Laput, a Ph.D. student in the Human-Computer Interaction Institute (HCII), suggesting totally new uses for smartwatches. "The hand is what people use to interact with the world."

Typically a smartwatch accelerometer would be used to detect when a person lifts an arm so the screen could be activated, or even to count footsteps. In these instances, the accelerometer only needs to take measurements about 100 times a second. However, when the researchers increased the sampling frequency to 4,000 a second, 4 kHz, they discovered that it acted like a vibrational microphone.

But instead of detecting sounds transmitted through the air, it works with the body to detect bio-acoustic signals.

Normally, "ViBand isn't just a way to control your smartwatch," said Chris Harrison, assistant professor of human-computer interaction at Carnegie Mellon. "It enables you to augment your arm. It's a powerful interface that's always available to you."

Incorporating the ViBand technology into a watch would allow it to recognize if someone is tapping on their forearm, palm of their hand or the back of their hand. It would detect finger flicks, scratches and other motions. It would even sense if a person is holding various mechanical and electrical tools, such as an electric toothbrush, power drill or handsaw. Each body tap, device or activity has distinctive bio-acoustic signals.

To increase the frequency of the accelerometer's sampling rate, the research team developed a custom kernel, which is at the core of the smartwatch's operating system. This was the only modification they needed to perform the software update, according to Laput.

In demonstrations, the team displayed potential apps for ViBand, which included using hand gestures in the area around the watch to control apps on the watch. Other gestures could be used to control remote devices for lights or a TV, as well as other appliances connected via the Internet of Things.

The team also showed it could be used for object-aware apps, such as monitoring meal preparations or providing visual feedback while tuning an acoustic guitar.

One other concept they showed off was an application that employed what they call a vibro-tag – a small object that emits inaudible, structured vibrations containing data. A vibro-tag could be placed on a person's office door, for example, to transmit information about office hours or alternative contact information.

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Find Free Electronics Datasheets