Playing the piano with a robotic third thumb: assessing constraints of human augmentation

Abstract

Contemporary robotics gives us mechatronic capabilities for augmenting human bodies with extra limbs. However, how our motor control capabilities pose limits on such augmentation is an open question. We developed a Supernumerary Robotic 3rd Thumbs (SR3T) with two degrees-of-freedom controlled by the user’s body to endow them with an extra contralateral thumb on the hand. We demonstrate that a pianist can learn to play the piano with 11 fingers within an hour. We then evaluate 6 naïve and 6 experienced piano players in their prior motor coordination and their capability in piano playing with the robotic augmentation. We show that individuals’ augmented performance with the SR3T could be explained by our new custom motor coordination assessment, the Human Augmentation Motor Coordination Assessment (HAMCA) performed pre-augmentation. Our work demonstrates how supernumerary robotics can augment humans in skilled tasks and that individual differences in their augmentation capability are explainable by their individual motor coordination abilities.

Swarm Body: Embodied Swarm Robots

“…Most studies on virtual and robot embodiments have focused on body configuration similar to the human biological body. Recently, researchers have begun to explore the embodiment of Supernumerary Robotic Appendages (SRAs) or unconventional body parts, such as additional limbs [58,71,76], hands [24,77], fingers [51,61], joints [36], and tails [20]. The embodiment of an SRA or an unconventional body has the potential to augment human abilities and interactions with the environment, which are currently limited by the innate body configuration.…”

A Model of Multi-Finger Coordination in Keystroke Movement

“…However, most of these studies merely observe and analyze the coordination phenomena, offering little guidance for practical engineering applications. Additionally, deterministic motion coordination relationships are needed in areas such as limb coordination, muscle energy consumption monitoring, and robotic adaptive control [ 14 , 15 , 16 ]. Given the interconnected tendons of the ring finger and adjacent fingers, with the middle and ring fingers exhibiting greater motion overflow characteristics, this study focuses on their coordinated keystroke actions in professional pianists, capturing these multi-finger coordinated movements and establishing a mathematical model.…”

Impact of supplementary sensory feedback on the control and embodiment in human movement augmentation

“…Several SRL prototypes are currently employed in laboratory environments, most of them in the form of additional fingers 18 – 22 or arms 23 – 25 . They are usually controlled by tracking movement of other limbs not directly involved in the task 19 , 26 , by reading user’s postural adjustments to trigger robotic support 27 , 28 or by employing dedicated interfaces such as joysticks 20 .…”