The StarLight system has a variety of potential applications, including virtual reality without on-body controllers and real-time health monitoring. The new system builds upon the team’s previous work, called license, and reduces the number of intrusive sensors while overcoming furniture blockage and supporting user mobility.
To create the system, the team studied the use of purely ubiquitous light around us to track users' behavior, without any cameras, on body devices or electromagnetic interference. The team reconstructed a user 3-D skeleton by leveraging the light emitted from LED panels on the ceiling and only 20 light sensors on the floor. The system can track the user's skeleton as he or she moves around a room with furniture and other objects.
"We're turning light into a ubiquitous sensing medium that tracks what we do and senses how we behave," said Xia Zhou, an assistant professor of computer science and co-director of the DartNets (Dartmouth Networking and Ubiquitous Systems) Lab. "Our new work demonstrates a new, unobtrusive sensing paradigm exploiting the light around us. It addresses several key practical issues of light-based sensing, including the furniture blockage, reliance on a large number of light sensors and user mobility. It pushes the vision of light sensing closer to practice.
According to the team’s experiments, StarLight achieves 13.6° mean angular error for five body joints and reconstructs a mobile skeleton at a high frame rate (40 FPS), enabling a new sensing paradigm to expand today’s mobile sensing behavioral monitoring.
"Light is everywhere and we're making it very smart," said Zhou. "Imagine a future where light knows and responds to what we do. We can naturally interact with surrounding smart objects, such as drones and smart appliances and play games, using purely the light around us. It can also enable a new passive health and behavioral-monitoring paradigm to foster healthy lifestyles or identify early symptoms of certain diseases. The possibilities are unlimited."