Consumer Peripherals

Video: MIT Teaches Artificial Intelligence to Do Household Tasks

30 May 2018

Everyone is busy. You have work. You have family. You have things to do and places to go. Household chores sometimes get pushed to the back burner because no one wants to do them in the first place and second, there simply isn’t time.

But what if a robot equipped with powerful artificial intelligence could be programmed to do those chores for you?

MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) is working on teaching machines to do a wider range of tasks around the house.

The technology, called VirtualHome, simulates detailed household tasks and then has artificial “agents” execute them, opening up the ability of one day teaching robots to do such tasks.

MIT CSAIL tested more than 3,000 different programs of various activities such as making coffee, opening a cabinet or grabbing a cup. Of these programs, the MIT technology can execute 1,000 separate sets of actions in eight different scenes, which include living room, kitchen, dining room, bedroom and home office.

“Describing actions as computer programs has the advantage of providing clear and unambiguous descriptions of all the steps needed to complete a task,” said Xavier Puig, an MIT CSAIL Ph.D. student who led the research. “These programs can instruct a robot or a virtual character, and can also be used as a representation for complex tasks with simpler actions.”

How it Works

The team collected verbal descriptions of household activities and then translated them into simple code. A program like this for a task such as “watch TV” might include steps like: walk to the TV, switch on the TV, walk to the sofa and sit on the sofa.

Once the programs were created, the MIT team fed them to the VirtualHome 3D simulator to be turned into videos. A virtual agent would then be able to execute the tasks defined by the programs, whether it was watching TV, placing a pot on the stove or turning a toaster on and off.

Beyond just a system for training robots to do chores, the system becomes a large database of household tasks described using natural language. MIT CSAIL said companies such as Amazon are working on robotic systems at home that could eventually use data like this to train their models to do more complex tasks.

The researchers hope to train the robots to use actual videos in the future as currently, they use simulation videos. This could mean simply watching a YouTube video to understand how to accomplish a new task. The system is also working on implementing a reward-learning system in which the agent gets positive feedback when it does tasks correctly.

Read the full research paper.

To contact the author of this article, email

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Find Free Electronics Datasheets