Top Story

Robot combines vision and touch to learn the game of Jenga

In the basement of MIT's Building 3, a robot is carefully contemplating its next move. It gently pokes at a tower of blocks, looking for the best block to extract without toppling the tower, in a solitary slow-moving, yet surprisingly agile game of Jenga. The robot is equipped with a soft-pronged gripper, a force-sensing wrist cuff, and an external camera, all of which it uses to see and feel the tower and its individual blocks. As the robot carefully pushes against a block, a computer takes in visual and tactile feedback from its camera and cuff and compares these measurements to moves that the robot previously made. It also considers the outcomes of those moves -- specifically, whether a block, in a certain configuration and pushed with a certain amount of force, was successfully extracted or not. In real-time, the robot then "learns" whether to keep pushing or move to a new block, in order to keep the tower from falling. The researchers believe the tactile learning system can be used in applications beyond Jenga, especially in tasks that need careful physical interaction, including separating recyclable objects from landfill trash and assembling consumer products.

Visit Website | Image credit: MIT