Researchers from the University of Texas at Arlington created a new way to help robots learn and navigate foreign environments. In order for robots to be able to navigate built environments, the machines need to be able to understand and make decisions about their surroundings.
The inspiration for this project came from a project assistant professor William Beksi was interning for the iRobot. This company was interested in using deep learning to train robots to learn about objects. But doing this requires a large amount of data including millions of photos and videos of rooms. When they gathered this data, none of the images were shot from the Roomba’s vantage point. Training the robots with images from a human perspective failed. So they were interested in creating algorithms that enable machines to learn from their interactions with the physical world and acquire the skills needed to do high-level tasks. Years later, this inspired Beski to find a way to help robots learn.
Examples of 3D point clouds synthesized by the progressive conditional generative adversarial network (PCGAN) for an assortment of object classes. Source: William Beksi, UT Arlington
Manual approaches to robot training are expensive, require a 360-degree camera and custom software to stitch the images together. In the end, this method is too slow to succeed.
When developing the new method, the team turned to a form of deep learning called generative adversarial networks (GANs). With GANs, two neural networks compete with each other until the generator of new data can fool the discriminator. Once trained, the network would be able to help create an infinite number of possible rooms or outdoor environments.
For their study, the team created a PCGAN. This is the first conditional generative adversarial network to create dense colored point clouds in unsupervised mode. The network is capable of learning from a training set and mimicking 3D data distribution to produce colored point clouds with fine details at multiple resolutions.
To test their method, the team fed the network an array of shapes, including chairs, tables, sofas, airplanes and motorcycles. The network allowed the team to access a near-infinite number of possible versions of set objects to train the algorithm. During training, the algorithm first learns basic object structure at a low resolution and gradually builds up toward high-level details. Eventually, the network was able to generate 5,000 random samples for each class and perform an evaluation with a number of different methods. It also evaluated point cloud geometry and color with a variety of common metrics. The results showed that PCGAN is capable of synthesizing high-quality point clouds for an array of object classes.
The team is also working on a problem called “sim2real”, in which an algorithm looks at how to quantify differences and make simulations more realistic by capturing the physics of the scene and using ray or photon tracing. The next step is to deploy the software on a robot and see how it works in relation to the sim2real domain gap.
This project is one step toward the overall goal of generating synthetic scenes of indoor environments for robot perception.
The paper on this research can be accessed here.
