Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, use sparse trial-and-error to learn multiple tasks despite their intricate tendon-driven anatomies—which are particularly hard to control because they are simultaneously nonlinear, under-determined, and over-determined. We demonstrate—for the first time in simulation and hardware—how a model-free, open-loop approach allows few-shot autonomous learning to produce effective movements in a 3-tendon 2-joint limb. We use a short period of motor babbling (to create an initial inverse map) followed by building functional habits by reinforcing high-reward behavior and refinements of the inverse map in a movement’s neighborhood. This biologically-plausible algorithm, which we call G2P (General-to-Particular), can potentially enable quick, robust and versatile adaptation in robots as well as shed light on the foundations of the enviable functional versatility of organisms.

Biological organisms learn from interactions with their environment throughout their lifetime. For artificial systems to successfully act and adapt in the real world, it is desirable to similarly be able to learn on a continual basis. This challenge is known as lifelong learning, and remains to a large extent unsolved. In this perspective article, we identify a set of key capabilities that artificial systems will need to achieve lifelong learning. We describe a number of biological mechanisms, both neuronal and non-neuronal, that help explain how organisms solve these challenges, and present examples of biologically inspired models and biologically plausible mechanisms that have been applied to artificial intelligence systems in the quest towards development of lifelong learning machines. We discuss opportunities to further our understanding and advance the state of the art in lifelong learning, aiming to bridge the gap between natural and artificial intelligence.

In addition to a vestibular system, birds uniquely have a balance-sensing organ within the pelvis, called the lumbosacral organ (LSO). The LSO is well developed in terrestrial birds, possibly to facilitate balance control in perching and terrestrial locomotion. No previous studies have quantified the functional benefits of the LSO for balance. We suggest two main benefits of hip-localized balance sense: reduced sensorimotor delay and improved estimation of foot-ground acceleration. We used system identification to test the hypothesis that hip-localized balance sense improves estimates of foot acceleration compared to a head-localized sense, due to closer proximity to the feet. We built a physical model of a standing guinea fowl perched on a platform, and used 3D accelerometers at the hip and head to replicate balance sense by the LSO and vestibular systems. The horizontal platform was attached to the end effector of a 6 DOF robotic arm, allowing us to apply perturbations to the platform analogous to motions of a compliant branch. We also compared state estimation between models with low and high neck stiffness. Cross-correlations revealed that foot-to-hip sensing delays were shorter than foot-to-head, as expected. We used multi-variable output error state-space (MOESP) system identification to estimate foot-ground acceleration as a function of hip-and head-localized sensing, individually and combined. Hip-localized sensors alone provided the best state estimates, which were not improved when fused with head-localized sensors. However, estimates from head-localized sensors improved with higher neck stiffness. Our findings support the hypothesis that hip-localized balance sense improves the speed and accuracy of foot state estimation compared to head-localized sense. The findings also suggest a role of neck muscles for active sensing for balance control: increased neck stiffness through muscle co-contraction can improve the utility of vestibular signals. Our engineering approach provides, to our knowledge, the first quantitative evidence for functional benefits of the LSO balance sense in birds. The Urbina-Meléndez et al.Hip Balance-Sense Implications Bipedal Robots findings support notions of control modularity in birds, with preferential vestibular sense for head stability and gaze, and LSO for body balance control,respectively. The findings also suggest advantages for distributed and active sensing for agile locomotion in compliant bipedal robots.

Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendondriven robot. We implemented two versions of the Generalto-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two joints and three tendons: one with and one without kinematic feedback. As expected, feedback improved performance in simulation and hardware. However, we see these improvements even in the presence of sensory delays of up to 100 ms and when experiencing substantial contact collisions. Importantly, feedback accelerates learning and enhances G2P's continual refinement of the initial inverse map by providing the system with more relevant data to train on. This allows the system to perform well even after only 60 seconds of initial motor babbling.