Industrial Electronics

Watch: Robots can use sound to analyze objects

14 August 2020

Researchers from Carnegie Mellon University found that robot perception is improved when robots can hear. Until now, robots have depended on sight and touch for perception, but their perception is not always accurate. By adding hearing to a robot’s list of senses, robots can be more precise when sorting items.

Carnegie Mellon University researchers devised an apparatus called Tilt-Bot to build a collection of actions, video and sound to improve robot perception. Source: Carnegie Mellon UniversityCarnegie Mellon University researchers devised an apparatus called Tilt-Bot to build a collection of actions, video and sound to improve robot perception. Source: Carnegie Mellon University

Sounds help robots differentiate between objects, determine what action caused a sound and predict the physical properties of objects. Robots that used sound successfully also classified objects 76% of the time.

For the study, the team created a large dataset of video and audio of 60 common objects. They recorded the objects moving around a tray and crashing into the sides. They cataloged 15,000 interactions with the objects.

The interactions were captured with Tilt-Bot, a square tray attached to the arm of the Sawyer robot. With Tilt-Bot, the team placed an object on the tray and let the Sawyer robot spend a few hours moving the tray in random directions with varying levels of tilt. Cameras and microphones recorded the actions. The team also collected data with Sawyer pushing objects around on a surface without the tray.

The team also studied how robots gather information from sound. For example, the robot used sound to estimate the amount of granular material by shaking the container. The robot could use what it learned about the sound of a set of objects to make predictions about the physical properties of unseen objects.

This research was presented at the virtual Robotics Science and Systems Conference.



Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Advertisement
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Advertisement
Find Free Electronics Datasheets
Advertisement