MIT engineers have developed a new virtual-reality training system for drones that enables a vehicle to “see” a rich, virtual environment while flying in an empty physical space. Source: William Litant
Drones are finding their way into more industries every day. Drones are being used to refuel military planes in flight, deliver mail, rescue people who are stranded at sea, take selfies and more. While there are a lot of amazing things that drones are doing, the drones need to be properly trained. MIT researchers have developed a new training method using virtual reality (VR) headsets, but instead of humans wearing the headsets, the drones are wearing them.
Currently, drones are being trained by humans, who are manually flying and controlling them through mazes and enclosed testing grounds. These testing grounds are set up for the drone to practice running through before it is put into the real world. The grounds have large nets set up to catch any drones that have gone off the path. While this method does work, it can be very costly. A drone can be destroyed or broken if it goes off the path or runs into any part of the maze. Every time that a drone is broken or destroyed, it can cost hundreds for repairs or a replacement.
MIT researchers wanted to find a way to make the training process cheaper and more streamlined. Sertac Karaman, associate professor of aeronautics and astronautics at MIT, found his inspiration in a new sport: competitive drone racing. These drones are remotely controlled by humans and race as fast as they can to out-fly each other in extreme mazes that have many obstacles. While watching these drones race each other, Karaman wondered if there was a way to train these drones to fly just as well, if not faster on their own, without humans controlling them.
Karaman and his team of colleagues created a VR headset for drones called Flight Goggles. Flight Goggles allow drones to see and attempt to navigate through a virtual environment that is based on a real-life environment while flying through an empty training space.
Fast drones have to process visual information quite quickly. The VR training system consists of a motion capture system, imaging rendering program and other imaging processing electronics. These technologies allow the researchers to transmit realistic virtual images straight to the drone’s headset. The drones can process the VR images at 90 frames per second. The team built circuit boards that have a powerful embedded supercomputer that has an inertial measurement unit and a camera, so the drone can process images quickly when traveling at a high speed. All of this tech can fit on the small, 3D-printed, nylon and carbon-fiber-reinforced drone frame.
The researchers can put any VR scene or layout that they want into the training system, so the team can make any area into a virtual environment. They are considering mapping and imaging MIT’s Stata Center to train drones to eventually fly around the center with no human control.
To test the Flight Goggles, the researchers put the VR-wearing headsets in a hangar-like area at MIT’s drone testing facility. The facility is lined with motion capture cameras that show the researchers how the drone is doing frame by frame. There were many tests that the researchers ran, including one where the drone had to fly through a window twice its size. Over 10 flights, the drone flew through the virtual window at 2.3 meters per seconds with 361 successful flights through the window and only three crashes. In the final test, the researchers took the headset off the drone and let it attempt to fly through a real window on its own. The drone was able to successfully fly 119 flights and only crashed or needed human intervention six times.
Other than flying through environments, the system can also be used to train drones to interact with humans. To do this training, the team would split the training area in half. On one side would be a human dressed in a motion tracking suit, with the drone on the other side being fed the virtual image of the human. The drone can “see” the human as it is flying around the facility, but it isn’t actually interacting with them.
