Robots use Cornell's RHyME AI to learn new skills by watching just one video

In context: Teaching robots new skills has traditionally been slow and painstaking, requiring hours of step-by-step demonstrations for even the simplest tasks. If a robot encountered something unexpected, like dropping a tool or facing an unanticipated obstacle, its progress would often grind to a halt. This inflexibility has long limited the practical use of robots in environments where unpredictability is the norm.

Researchers at Cornell University are now charting a new course with RHyME, an artificial intelligence framework that dramatically streamlines robot learning. An acronym for Retrieval for Hybrid Imitation under Mismatched Execution, RHyME enables robots to pick up new skills by watching a single demonstration video. This is a sharp departure from the exhaustive data collection and flawless repetition previously required for skill acquisition.

The key advance with RHyME is its ability to overcome the challenge of translating human demonstrations into robotic actions. While humans naturally adapt their movements ...