Proximal Policy Optimization With Time-Varying Muscle Synergy for the Control of an Upper Limb Musculoskeletal System

Liu, Rong; Wang, Jiaxing; Chen, Yaru; Liu, Yin; Wang, Yongxuan

doi:10.1109/tase.2023.3254583

Cited by 4 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another study [30] developed a deep Q networkbased controller for a musculoskeletal arm model, employing a phased target-learning framework and human-inspired noise for stable and efficient exploration of the solution space. Complex reinforcement learning algorithms such as Deep Deterministic Policy Gradient [67], proximal policy optimisation [68], and Soft Actor-Critics [85] have also shown promising control results.…”

Section: Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

Utilising redundancy in musculoskeletal systems for adaptive stiffness and muscle failure compensation: a model-free inverse statics approach

Almanzor,

Sugiyama,

Abdulali

et al. 2024

Bioinspir. Biomim.

View full text Add to dashboard Cite

Vertebrates possess a biomechanical structure with redundant muscles, enabling adaptability in uncertain and complex environments.Harnessing this inspiration, musculoskeletal systems offer advantages like variable stiffness and resilience to actuator failure and fatigue.Despite their potential, the complex structure presents modelling challenges that are difficult to explicitly formulate and control. This difficulty arises from the need for comprehensive knowledge of the musculoskeletal system, including details such as muscle arrangement, and fully accessible muscle and joint states.Whilst existing model-free methods do not need explicit formulations, they also underutilise the benefits of muscle redundancy. Consequently, they necessitate retraining in the event of muscle failure and require manual tuning of parameters to control joint stiffness limiting their applications under unknown payloads. Presented here is a model-free local inverse statics controller for musculoskeletal systems, employing a feedforward neural network trained on motor babbling data. Experiments with a musculoskeletal leg model showcase the controller's adaptability to complex structures, including mono and bi-articulate muscles. The controller can compensate for changes such as weight variations, muscle failures, and environmental interactions, retaining reasonable accuracy without the need for any additional retraining.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

“…The only prior work that undergoes this retraining process is constrained to joint space control [65]. While reinforcement learning approaches have proven successful in highly redundant musculoskeletal systems [66][67][68], they often mandate complex reward function design and laborious retraining after muscle failure.…”

Section: Introductionmentioning

confidence: 99%