Have you ever walked down the street and been annoyed by someone walking on the “wrong” side of the sidewalk? That person has broken one of the unspoken sidewalk rules. Generally, the rules are to keep right, pass left, maintain a respectable distance from the people around you, be ready to weave to change course for oncoming obstacles and keep up a steady walking pace. Engineers at MIT have designed an autonomous robot that has “socially aware navigation” that keeps pace with foot traffic and observes the unspoken sidewalk code.
Drive tests were performed inside MIT’s Stata Center and the robot successfully avoided collisions and kept up with the average flow of pedestrians. The robot resembles a knee-high kiosk on wheels.
"Socially aware navigation is a central capability for mobile robots operating in environments that require frequent interactions with pedestrians," said Yu Fan "Steven" Chen, who led the work as a former MIT graduate student and is the lead author of the study. "For instance, small robots could operate on sidewalks for package and food delivery. Similarly, personal mobility devices could transport people in large, crowded spaces, such as shopping malls, airports, and hospitals."
In order for a robot to make its way through a heavily trafficked environment on its own, it must solve four main challenges: localization (knowing where it is in the world), perception (recognizing the surroundings), motion planning (identifying the optimal path to its destination) and control (physically executing the desired path).
Chen and his team used standard approaches to solve the problems of localization and perception. For perception, the team outfitted the robot with off-the-shelf sensors, like webcams, a depth sensor, and high-resolution lidar sensor. For localization, the team used open-source algorithms to map the robot’s environment and determine the position. To control the robot they employed standard methods used to drive autonomous ground vehicles.
"The part of the field that we thought we needed to innovate on was motion planning," Everett said. "Once you figure out where you are in the world and know how to follow trajectories, which trajectories should you be following?"
That is a difficult problem with the robot, especially in a pedestrian-heavy environment where the individual paths are often hard to predict. The roboticists found a solution for this by taking a trajectory-based approach where they programed the robot to compute an optimal path that accounts for everyone’s desired trajectories. The trajectories must be inferred from sensor data because people don’t tell the robot where they are trying to go.
"But this takes forever to compute. Your robot is just going to be parked, figuring out what to do next, and meanwhile, the person's already moved way past it before it decides 'I should probably go to the right,'" Everett said. "So that approach is not very realistic, especially if you want to drive faster."
Other researchers have used faster “reactive-based” approaches where the robot is programmed with a simple model using geometry or physics to quickly compute a path that avoids collisions. The problem with these reactive-based approaches is the unpredictability of human nature. People rarely stick to a straight, geometric path, and they tend to weave and wander and veer off to greet a friend or grab a coffee. In these unpredictable environments, the robots tend to collide with people or look like they are being pushed by avoiding people excessively.
"The knock on robots in real situations is that they might be too cautious or aggressive," Everett said. "People don't find them to fit into the socially accepted rules, like giving people enough space or driving at acceptable speeds, and they get more in the way than they help."
The research team found a way to reduce the limitations by enabling the robot to adapt to unpredictable pedestrian behavior while moving with the flow and following typical social codes of pedestrian conduct.
They used reinforcement learning where they performed computer simulations to train a robot to take certain paths, given the speeds and trajectory of other objects in the environment. The team incorporated social norms into the offline training pace and they encouraged the robot in simulations to pass on the right and penalized the robot when it passed on the left.
"We want it to be traveling naturally among people and not be intrusive," Everett said. "We want it to be following the same rules as everyone else."
The advantage to reinforcement learning is that the researchers can perform the training scenarios that takes extensive time and computing power, offline. When the robot is trained in simulation, the researchers can program it to carry out the optimal paths, identified the simulations and the robot can recognize a scenario in the real world.
The researchers enabled this robot to assess the environment and adjust the path every one-tenth of a second. The robot can continue to roll through a hallway at a typical walking speed of 1.2 meters per second without having to pause to reprogram.
"We're not planning an entire path to the goal — it doesn't make sense to do that anymore, especially if you're assuming the world is changing," Everett said. "We just look at what we see, choose a velocity, do that for a tenth of a second, then look at the world again, choose another velocity, and go again. This way, we think our robot looks more natural and is anticipating what people are doing."
The robot is still being tested at MIT. There, the robot was able to drive autonomously for 20 minutes at a time. It rolled smoothly through pedestrian flow and kept to the right of the hallways while occasionally passing people on the left.
A paper on this research will be presented at the IEEE Conference on Intelligent Robots and Systems in September.