This image shows targets in the corner of a TV screen. Each target rotates around its corresponding function. The user matches the rotational movement with any object, or part of their body, to create a coupling and activate the control. (Lancaster University)
New gesture control technology that can turn ordinary objects into remote controls could change how people interact with televisions and other screens. This could end the annoying search for a lost remote in the black hole that is the couch cushions.
The ‘Matchpoint’ technology only requires a webcam. It works by displaying moving targets that orbit a small, circular widget in the corner of the screen. These targets have different functions like volume, changing a channel or viewing a menu. Users synchronize the direction of movement of the target with their hand, head or with an object in order to achieve "spontaneous spatial coupling" that activates the function.
Unlike existing gesture control technology, this software doesn’t look for a specific body part that it has been trained to identify, like a hand. Lancaster’s technology looks for rotating movement, so it doesn’t require calibration or the software to have prior knowledge of objects. This provides more flexibility and ease for users because it works even when their hands are full and while standing or slouching on the sofa.
Users don’t need to learn specific commands in order to activate different functions. The user is able to decouple at will.
When adjusting volume or channel selection, sliders appear on the screen. The users move their hand, head, or object as required in the direction indicated by the slider to change the volume or find the desired brand.
Technology can be used with other screens, like YouTube tutorials. Videos like baking a cake tutorial could be paused and rewound on tablets and computers easily without user having to stop what they are doing.
Multiple pointers can be created that allow more than one user to point at drawings or pictures on interactive whiteboards simultaneously. Matchpoint allows users to manipulate images on whiteboards using two hands to zoom in and out and rotate images.
Along with short-term couplings, users can link stationary objects to specific controls. Even when left for prolonged periods, the objects will retain their control function. A mug sitting on a table could change a track on a music player when moved left or right, for example. Objects can lose their coupling with controls simply by removing them from the camera’s view.
Christopher Clarke, Ph.D. student at Lancaster University’s School of Computing and Communications and developer of Matchpoint, said, “Spontaneous spatial coupling is a new approach to gesture control that works by matching movement instead of asking the computer to recognize a specific object. Our method allows for a much more user-friendly experience where you can change channels without having to put down your drink, or change your position, whether that is relaxing on the sofa or standing in the kitchen following a recipe. Everyday objects in the house can now easily become remote controls so there are no more frantic searches for remote controls when your favorite program is about to start on another channel and now everyone in the room has the 'remote'. You could even change the channel with your pet cat."
Researchers believe that Matchpoint can be used as an accessibility tool for people who can’t use traditional pointers.
A paper on this new technology will be presented at the UIST2017 conference in Quebec City this month.
