Meet PianoMime, the Robot Pianist!
2 min readJul 27, 2024
In this post, we introducing PianoMime, a super cool framework that’s teaching robots to play the piano using videos from the internet.
This is the result of a research focusing on Imitation Learning, Reinforcement Learning, Dexterous Manipulation, Learning from Observations done by Cheng Qian, Julen Urain, Kevin Zakka and Jan Peters published in the following paper on 25th of July:
What’s PianoMime All About?
The idea is simple yet groundbreaking: teach a robot to play any song on the piano by watching YouTube videos of professional pianists. Yep, you read that right. The internet is full of videos of piano maestros performing, and PianoMime taps into this treasure trove to train its robot.
How Does It Work?
The process is divided into three main phases:
- Data Preparation: The team gathers videos from YouTube, especially those with a clear top-view of the…
