A robot capable of locating lost items on command has been developed by a team from the Technical University of Munich (TUM).
To enable the robot, which resembles a broomstick on wheels with a camera mounted at the top, to locate lost objects, the team combined knowledge from the internet with a spatial map of the robot’s surroundings.
Source: TUM
The team explained that when tasked with finding a pair of glasses misplaced in a kitchen, the robot looked around and built a three-dimensional image of the room. While the camera provides two-dimensional images, these pixels also feature depth information, thus creating a spatial map of the environment that is frequently updated and reportedly accurate to the centimeter.
Because it has been taught to understand its surroundings, the robot can understand, for instance that a table or windowsill can be used to briefly set down a pair of glasses, while a stovetop or a sink are likely not suitable.
"The language model captures the relationships between the objects and we convert this information into the robot's language," the team explained.
Specifically, two-digit numbers will appear on the 3D map of the environment, continuously recalculating the likelihood that the object in question is located there. During trials, the team found that the robot searched the probable locations almost 30% more efficiently than if it searched randomly.
Using artificial intelligence (AI) in both image recognition and through the use of a language model, the robot can remember previous images and compare them with new images of its surroundings. When a new object appears in the kitchen, for instance, the robot recognizes the change with a 95% degree of certainty and brands these areas as "highly probable" search locations.
The work is detailed in the article, “Where Did I Leave My Glasses? Open-Vocabulary Semantic Exploration in Real-World Semi-Static Environments,” which appears in the journal IEEE Robotics and Automation Letters.
For more on the robot, watch the accompanying video that appears courtesy of the Technical University of Munich.
