Watch Shimon the marimba-playing robot play along to jazz, reggae, and hip hop

Shimon is a marbima-playing robot with some real soul. This crazy little robot, created by Gil Weinberg at the Georgia Tech Center for Music Technology, can listen to the other players around it and play out little ditties in response to the music. In short, it’s the world’s best jazz and hip hop collaborator because, unlike humans, Shimon can never get drunk and forget the van keys back at that Taco Bell in Fresno.

“Most of what Shimon is playing is generated using a new process where he creates hundreds of melodies off line based on deep learning analysis of large musical data sets,” said Weinberg. “Then us humans (me and my students) choose melodies we like and orchestrate / structure them into songs. It’s a new form of robot-human collaboration, at least for us.”

In this video Shimon and crew play along to Dash Smith, an Atlanta-based rapper who freestyles while Shimon and you’ll also notice another Georgia Tech product, a robotic drumming prosthesis that gives the drummer the power of four Neil Perts.

Weinberg, Shimon’s human, is excited by the new developments.

“Still under development is the other new element – we are working letting Shimon analyze in real time the rhythm, melodies and semantic meaning of the free style rapper lyrics and use this analysis to drive Shimon’s improvisation. As you know we have explored mostly improvised music, starting with drum circles moving to Jazz, rock jam-bands, and African marimba bands,” said Weinberg. “We are now ready to move to the next frontier of real time collaborative improvisation – free style rapping, where the hope is that the rapper will be influenced by what Shimon is coming up with and vice versa.”