A new AI system that can see human movements through walls has been developed by a research team from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).
The new system is called RF-Prose. It uses AI to teach wireless devices to detect and sense a person’s movement or position through a wall. The system uses a neural network to detect and analyze the radio signals that bounce off people’s bodies, and then creates a dynamic stick figure that can walk, stop, sit and move like a human.
RF-Prose could be used to monitor diseases like Parkinson’s or multiple sclerosis to better understand the development of these diseases and help doctors adjust medications for each individual patient. The system could also help elderly people live more independently by adding an extra level of security that monitors accidents and activity changes in the person’s life. RF-Prose could also be used for new video games and in search-and-rescue missions.
This is Mingmin Zhao, pictured right. Source: Jason Dorfman, MIT CSAIL
"We've seen that monitoring patients' walking speed and ability to do basic activities on their own gives healthcare providers a window into their lives that they didn't have before, which could be meaningful for a whole range of diseases," says Professor Dina Katabi , who led the team and co-wrote a new paper about the project. "A key advantage of our approach is that patients do not have to wear sensors or remember to charge their devices."
The researchers faced one main challenge when developing RF-Prose. Most neural networks are trained using data that has been labeled by humans, but RF-Prose uses radio frequencies, which are nearly impossible to label by hand. They solved the problem by collecting examples of radio frequencies bouncing off human bodies with a wireless device and a camera. Thousands of images of people in various activities, like walking, waiting for an elevator and more, were taken.
A stick figure of the human’s general pose was then created. The stick figures and images were shown to the neural network with a corresponding radio signal. These examples allowed the system to learn which radio signals were associated with the stick figures and positions.
After training, the RF-Prose was able to estimate a person’s posture and movements with only the radio waves that bounce off of people’s bodies.
The system was never trained to read through walls because cameras can’t see through walls. RF-Prose taught itself to detect and analyze the radio signals coming through a wall and it was able to figure out what a person’s position is on the other side of the wall.
The system was also able to recognize a specific person out of a 100-person line up with 83 percent accuracy. This is an important ability if the system is going to be used in a search-and-rescue mission.
The paper on this research will be presented at the Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City.
