Key pointsr It is often assumed that central pattern generators, which generate rhythmic patterns without rhythmic inputs, play a key role in the spinal control of human locomotion.r We propose a neural control model in which the spinal control generates muscle stimulations mainly through integrated reflex pathways with no central pattern generator.r Using a physics-based neuromuscular human model, we show that this control network is sufficient to compose steady and transitional 3-D locomotion behaviours, including walking and running, acceleration and deceleration, slope and stair negotiation, turning, and deliberate obstacle avoidance.r The results suggest feedback integration to be functionally more important than central pattern generation in human locomotion across behaviours.r In addition, the proposed control architecture may serve as a guide in the search for the neurophysiological origin and circuitry of spinal control in humans.Abstract Neural networks along the spinal cord contribute substantially to generating locomotion behaviours in humans and other legged animals. However, the neural circuitry involved in this spinal control remains unclear. We here propose a specific circuitry that emphasizes feedback integration over central pattern generation. The circuitry is based on neurophysiologically plausible muscle-reflex pathways that are organized in 10 spinal modules realizing limb functions essential to legged systems in stance and swing. These modules are combined with a supraspinal control layer that adjusts the desired foot placements and selects the leg that is to transition into swing control during double support. Using physics-based simulation, we test the proposed circuitry in a neuromuscular human model that includes neural transmission delays, musculotendon dynamics and compliant foot-ground contacts. We find that the control network is sufficient to compose steady and transitional 3-D locomotion behaviours including walking and running, acceleration and deceleration, slope and stair negotiation, turning, and deliberate obstacle avoidance. The results suggest feedback integration to be functionally more important than central pattern generation in human locomotion across behaviours. In addition, the proposed control architecture may serve as a guide in the search for the neurophysiological origin and circuitry of spinal control in humans.

Key pointsr Although the natural decline in walking performance with ageing affects the quality of life of a growing elderly population, its physiological origins remain unknown.r By using predictive neuromechanical simulations of human walking with age-related neuro-musculo-skeletal changes, we find evidence that the loss of muscle strength and muscle contraction speed dominantly contribute to the reduced walking economy and speed.r The findings imply that focusing on recovering these muscular changes may be the only effective way to improve performance in elderly walking.r More generally, the work is of interest for investigating the physiological causes of altered gait due to age, injury and disorders.Abstract Healthy elderly people walk slower and energetically less efficiently than young adults. This decline in walking performance lowers the quality of life for a growing ageing population, and understanding its physiological origin is critical for devising interventions that can delay or revert it. However, the origin of the decline in walking performance remains unknown, as ageing produces a range of physiological changes whose individual effects on gait are difficult to separate in experiments with human subjects. Here we use a predictive neuromechanical model to separately address the effects of common age-related changes to the skeletal, muscular and nervous systems. We find in computer simulations of this model that the combined changes produce gait consistent with elderly walking and that mainly the loss of muscle strength and mass reduces energy efficiency. In addition, we find that the slower preferred walking speed of elderly people emerges in the simulations when adapting to muscle fatigue, again mainly caused by muscle-related changes. The results suggest that a focus on recovering these muscular changes may be the only effective way to improve performance in elderly walking.

Self-selected walking speed is an important 1 aspect of mobility. Exoskeletons can increase walking 2 speed, but the mechanisms behind these changes and 3 the upper limits on performance are unknown. Human-4 in-the-loop optimization is a technique for identifying ex-5 oskeleton characteristics that maximize the benefits of 6 assistance, which has been critical to achieving large improvements in energy economy. In this study, we used 8 human-in-the-loop optimization to test whether large im-9 provements in self-selected walking speed are possible 10 through ankle exoskeleton assistance. Healthy participants 11 (N = 10) were instructed to walk at a comfortable speed 12 on a self-paced treadmill while wearing tethered ankle 13 exoskeletons. An algorithm sequentially applied different patterns of exoskeleton torque and estimated the speed-15 optimal pattern, which was then evaluated in separate trials. 16 With torque optimized for speed, participants walked 42% 17 faster than in normal shoes (1.83 m s −1 vs. 1.31 m s −1 ; 18 Tukey HSD, p = 4 × 10 −8 ), with speed increases ranging 19 from 6% to 91%. Participants walked faster with speed-20 optimized torque than with torque optimized for energy 21 consumption (1.55 m s −1 ) or torque chosen to induce 22 slow walking (1.18 m s −1 ). Gait characteristics with speed-23 optimized torque were highly variable across participants, 24 and changes in metabolic cost of transport ranged from a 25 31% decrease to a 78% increase, with a decrease of 2% 26 on average. These results demonstrate that ankle exoskele-27 tons can facilitate large increases in self-selected walking 28 speed, which could benefit older adults and others with 29 reduced walking speed. 30 Index Terms-exoskeleton gait assistance, human-in-31 the-loop optimization, walking speed 32 I. INTRODUCTION 33 E XOSKELETONS can improve some aspects of walking 34 performance and show promise for improving human 35 mobility overall. Fully-actuated lower-limb exoskeletons can 36 enable people with serious neurological injuries to walk again 37 [1], [2]. Exoskeletons that assist the hip, knee or ankle joints 38 individually tend to be more compact, but can still enhance 39 healthy and mildly-impaired gait. Recently, hip and ankle exoskeletons have been used to reduce metabolic energy 41 consumption during walking [3]-[8]. While improving energy 42 economy is important, other equally important dimensions of 43

Modeling human motor control and predicting how humans will move in novel environments is a grand scientific challenge. Researchers in the fields of biomechanics and motor control have proposed and evaluated motor control models via neuromechanical simulations, which produce physically correct motions of a musculoskeletal model. Typically, researchers have developed control models that encode physiologically plausible motor control hypotheses and compared the resulting simulation behaviors to measurable human motion data. While such plausible control models were able to simulate and explain many basic locomotion behaviors (e.g. walking, running, and climbing stairs), modeling higher layer controls (e.g. processing environment cues, planning long-term motion strategies, and coordinating basic motor skills to navigate in dynamic and complex environments) remains a challenge. Recent advances in deep reinforcement learning lay a foundation for modeling these complex control processes and controlling a diverse repertoire of human movement; however, reinforcement learning has been rarely applied in neuromechanical simulation to model human control. In this paper, we review the current state of neuromechanical simulations, along with the fundamentals of reinforcement learning, as it applies to human locomotion. We also present a scientific competition and accompanying software platform, which we have organized to accelerate the use of reinforcement learning in neuromechanical simulations. This “Learn to Move” competition was an official competition at the NeurIPS conference from 2017 to 2019 and attracted over 1300 teams from around the world. Top teams adapted state-of-the-art deep reinforcement learning techniques and produced motions, such as quick turning and walk-to-stand transitions, that have not been demonstrated before in neuromechanical simulations without utilizing reference motion data. We close with a discussion of future opportunities at the intersection of human movement simulation and reinforcement learning and our plans to extend the Learn to Move competition to further facilitate interdisciplinary collaboration in modeling human motor control for biomechanics and rehabilitation research

Controllers in robotics often consist of expertdesigned heuristics, which can be hard to tune in higher dimensions. It is typical to use simulation to learn these parameters, but controllers learned in simulation often don't transfer to hardware. This necessitates optimization directly on hardware. However, collecting data on hardware can be expensive. This has led to a recent interest in adapting dataefficient learning techniques to robotics. One popular method is Bayesian Optimization (BO), a sample-efficient black-box optimization scheme, but its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge to reduce dimensionality in a meaningful way, with a focus on bipedal locomotion. In previous work, we proposed a transformation based on knowledge of human walking that projected a 16-dimensional controller to a 1-dimensional space. In simulation, this showed enhanced sample efficiency when optimizing human-inspired neuromuscular walking controllers on a humanoid model. In this paper, we present a generalized feature transform applicable to nonhumanoid robot morphologies and evaluate it on the ATRIAS bipedal robot -in simulation and on hardware. We present three different walking controllers; two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.