Simple Kinematic Feedback Enhances Autonomous Learning in Bio-Inspired Tendon-Driven Systems

Abstract

Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendondriven robot. We implemented two versions of the Generalto-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two joints and three tendons: one with and one without kinematic feedback. As expected, feedback improved performance in simulation and hardware. However, we see these improvements even in the presence of sensory delays of up to 100 ms and when experiencing substantial contact collisions. Importantly, feedback accelerates learning and enhances G2P's continual refinement of the initial inverse map by providing the system with more relevant data to train on. This allows the system to perform well even after only 60 seconds of initial motor babbling.

Autonomous Control of a Tendon-driven Robotic Limb with Elastic Elements Reveals that Added Elasticity can Enhance Learning

“…In this paper, we studied how adding elastic elements affects autonomous learning in a two-joint three-tendons simulated limb (similar to [23], [25]) in the MuJoCo environment [32](Fig. 1.a).…”

“…G2P is a hierarchical autonomous learning algorithm that, on its lower-level, creates an inverse kinematics map using output kinematics collected from an initial random set of actuation commands (motor babbling). Systems that use an explicit kinematics model are, in general, easier to study and interpret, more data efficient and can generalize to a wider range of tasks; however, they can suffer from inaccuracies in the model especially during complex dynamical interactions (e.g., contact dynamics, injury to the body, or changes in the environment) [25], [23], [34], [35], [36], [37]. Systems that perform end-to-end learning (such as PPO), on the other hand, usually require larger number of samples to learn to perform a task, are harder to interpret due to their implicit modeling, and usually cannot generalize well across tasks [38], [39], [33], [40], [41].…”

“…The recorded kinematics are joint angles, angular velocities, and angular accelerations for both joints (a vector of 6 values). Next, we trained a Multi-Layer Perceptron (MLP) Artificial Neural Network (ANN; one hidden layer with 15 neurons; trained for 20 epochs; 80% training 20% validation; loss function: MSE, optimizer: ADAM) with kinematics as input and activations as output to form the inverse kinematics map (similar to [23], [25]). Finally, this inverse map was used to control the system to perform two tasks: Cyclical and Point-to-point movements.…”

“…For each joint, we calculate the error as the Root Mean Square Error (RMSE) of the difference between the joint angle and the desired angle in Radians. We disregard the error for the first 25% of the signal to make sure any initial condition effect is washed out [23], [25].…”

“…Thus, it can be challenging to find solutions that satisfy all the constraints imposed by tendons and by task specifications at the same time [23], [20]. Moreover, the fact that their actuators are not directly operating on the degrees of freedom (as is the case in joint-driven systems), makes it challenging to use an off the shelf controller (such as a simple PID setup) without having access to dynamical equations of the system or a forward or inverse kinematics model [25]. Also, these tendon-driven systems often require accurate modeling and control strategies for applications such as animation of lifelike figures [26], control of anatomical limbs to understand neurological conditions [27], [28], [29], or functional electrical stimulation of limbs (e.g., [30] or [31]).…”