Robots Learn from YouTube

Robots learn to perform tasks by "watching" YouTube videos.

Things would be a lot easier in robotics if robots could just learn from any of the millions of instructional videos on YouTube. The problem is that while robots can learn to recognize objects and patterns fairly well, they still can't interpret and act on the visual input. 

A research team from the University of Maryland, is in the very early stages of programing robots to take visual information from videos and turn it into action. The team's research is funded by DARPA's Mathematics of Sensing, Exploitation and Execution (MSEE) program, which aims to teach machines not only how to collect data, but also how to act on it. For  this particular study, the researchers have developed a system that allows their test robots to learn from a series of "how-to" cooking videos on YouTube. Based on what was shown in the videos, the robots were able to recognize, grab and manipulate the correct kitchen utensil or object and perform the demonstrated tasks with high accuracy and without additional human input or programming.

"The MSEE program initially focused on sensing, which involves perception and understanding of what's happening in a visual scene, not simply recognizing and identifying objects," said Reza Ghanadan, program manager in DARPA's Defense Sciences Offices. "We've now taken the next step to execution, where a robot process visual cues through a manipulation action-grammar module and translates them into actions." 

Another significant advance to come out of the research is the robots' ability to accumulate and share knowledge with other robots. They can store whatever they've learned from the videos and build on that knowledge to become better at handling certain tools or performing certain tasks. They can then share that information with other machines.

"This system allows robots to continuously build on previous learning -such as types of objects and grasps associated with them- which could have a huge impact on teaching and training," Ghanadan said. "Instead of the long and expensive process of programming code to teach robots to do tasks, this research opens the potential for robots to learn much faster, at much lower cost and, to the extent they are authorized to do so, share that knowledge with other robots. This learning-based approach is a significant step towards developing technologies that could have benefits in areas such as military repair and logistics."

So will we start to see advanced interconnected robots taking over the world in the near future, or will my teppanyaki chef be a four armed robot that always uses too much pepper?