VIDEO: Control Your Smartwatch without Touching It

A novel input method expands the input space to the back of the hand and the 3-D space above it. Source: Oliver DietzeFor anyone who owns a smart device, controlling it by pinches and spreads of the fingers on its screen is second-nature. But imagine if you could control it without actually touching it – by making the same finger movements on the back of your hand, or even in the empty space above it.

That’s the idea behind the novel technology developed by a team of researchers from the Max Planck Institute for Informatics in Germany, the University of Copenhagen and Aalto University in Helsinki. The researchers’ prototype, dubbed “WatchSense,” employs a depth sensor not unlike the Kinect game controller from an Xbox 360 – albeit much smaller. The sensor is worn on the user’s forearm, about 20 centimeters from a smartwatch. It works like a 3-D camera to capture and recognize movements in space, allowing for more expressive interactions than previous sensing devices could process.

One challenge the research team needed to solve was distinguishing between the thumb and index finger, which can occlude one another as they are moved within space. They trained their software system to recognize exact positions through machine learning. ”The most important thing is that we can not only recognize the fingers, but also distinguish between them," said researcher Srinath Sridhar, "which nobody else had managed to do before in a wearable form factor. We can now do this even in real time."

The technology also has been demonstrated with other touch-sensitive devices such as smartphones, smart TVs and augmented/virtual reality devices. Eventually, the researchers believe, depth sensors will be able to be made small enough to be integrated directly into a smartwatch itself. "We need something like WatchSense whenever we want to be productive while moving,” says Sridhar. “WatchSense is the first to enable expressive input for devices while on the move."