No abstract

Testing hypotheses of neuromuscular function during locomotion ideally requires the ability to record cellular responses and to stimulate the cells being investigated to observe downstream behaviors [1]. The inability to stimulate in free flight has been a long-standing hurdle for insect flight studies. The miniaturization of computation and communication technologies has delivered ultra-small, radio-enabled neuromuscular recorders and stimulators for untethered insects [2-8]. Published stimulation targets include the areas in brain potentially responsible for pattern generation in locomotion [5], the nerve chord for abdominal flexion [9], antennal muscles [2, 10], and the flight muscles (or their excitatory junctions) [7, 11-13]. However, neither fine nor graded control of turning has been demonstrated in free flight, and responses to the stimulation vary widely [2, 5, 7, 9]. Technological limitations have precluded hypotheses of function validation requiring exogenous stimulation during flight. We investigated the role of a muscle involved in wing articulation during flight in a coleopteran. We set out to identify muscles whose stimulation produced a graded turning in free flight, a feat that would enable fine steering control not previously demonstrated. We anticipated that gradation might arise either as a function of the phase of muscle firing relative to the wing stroke (as in the classic fly b1 muscle [14, 15] or the dorsal longitudinal and ventral muscles of moth [16]), or due to regulated tonic control, in which phase-independent summation of twitch responses produces varying amounts of force delivered to the wing linkages [15, 17, 18].

Reinforcement learning has emerged as a promising methodology for training robot controllers. However, most results have been limited to simulation due to the need for a large number of samples and the lack of automated-yet-safe data collection methods. Model-based reinforcement learning methods provide an avenue to circumvent these challenges, but the traditional concern has been the mismatch between the simulator and the real world. Here, we show that control policies learned in simulation can successfully transfer to a physical system, composed of three Phantom robots pushing an object to various desired target positions. We use a modified form of the natural policy gradient algorithm for learning, applied to a carefully identified simulation model. The resulting policies, trained entirely in simulation, work well on the physical system without additional training. In addition, we show that training with an ensemble of models makes the learned policies more robust to modeling errors, thus compensating for difficulties in system identification. The results are illustrated in the accompanying video.

No abstract

Abstract-We describe a computationally-expensive but very accurate approach to state estimation, which fuses any available sensor data with physical consistency priors. This is done by combining the advantages of recursive estimation and fixedlag smoothing: at each step we re-estimate the trajectory over a time window into the past, but also use a recursive prior obtained from the previous time step via internal simulation. We also incorporate a physics engine into the estimator, which makes it possible to adjust the state estimates so that the inferred contact interactions are consistent with the observed accelerations. The estimator can utilize contact sensors to improve accuracy, but even in the absence of such sensors it reasons correctly about contact forces. Estimation speed and accuracy are demonstrated on a 28-DOF humanoid robot (Darwin) in a walking task. Timing tests and leave-one-out cross-validation show that the proposed approach can be used in real-time and is substantially more accurate than the EKF, without any over-fitting.