Model Predictive-Actor Critic Reinforcement Learning for Dexterous Manipulation

Omer, M.K.; Ahmed, Rami; Rosman, Benjamin; Babikir, Sharief Fadul

doi:10.1109/iccceee49695.2021.9429677

Cited by 8 publications

(4 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model-Predictive Control (MPC), an offline learning approach, and online planning, which may be thought of as a model-based RL method, are combined to create MPC-SAC -25 [109]. Similarly, model-based methods such as time-varying linear-Gaussian [102] and deep neural networks [110].…”

Section: Reinforcement Learning (Rl)mentioning

confidence: 99%

Review on human‐like robot manipulation using dexterous hands

Sampath

Wang

Wu³

et al. 2023

Cognitive Comp and Systems

View full text Add to dashboard Cite

In recent years, human hand‐based robotic hands or dexterous hands have gained attention due to their enormous capabilities of handling soft materials compared to traditional grippers. Back in the earlier days, the development of a hand model close to that of a human was an impossible task but with the advancements made in technology, dexterous hands with three, four or five‐fingered robotic hands have been developed to mimic human hand nature. However, human‐like manipulation of dexterous hands to this date remains a challenge. Thus, this review focuses on (a) the history and motivation behind the development of dexterous hands, (b) a brief overview of the available multi‐fingered hands, and (c) learning‐based methods such as traditional and data‐driven learning methods for manipulating dexterous hands. Additionally, it discusses the challenges faced in terms of the manipulation of multi‐fingered or dexterous hands.

show abstract

Section: Reinforcement Learning (Rl)mentioning

confidence: 99%

Review on human‐like robot manipulation using dexterous hands

Sampath

Wang

Wu³

et al. 2023

Cognitive Comp and Systems

View full text Add to dashboard Cite

show abstract

“…Hence, most researchers focused on enhancing the sample efficiency but from various angles. Some of the researchers focus on the algorithm itself and test only in simulation (Popov et al, 2017 ; Haarnoja et al, 2019 ; Omer et al, 2021 ). Popov et al ( 2017 ) decouples the update from the frequency of interaction and trades off between the exploration and the exploitation by defining certain starting states and shaping reward effortfully.…”

Section: Dexterous Manipulation For Multi-fingered Robotic Hands With...mentioning

confidence: 99%

“…Haarnoja et al ( 2019 ) improved the SAC for accelerating training and improving stability. Omer et al ( 2021 ) present MPC-SAC combining the Model-Predictive Control (MPC) which is an offline learning method with online planning, which can be seen as a model-based RL method. Similarly, model-based methods are also adopted in (Kumar et al, 2016 ; Nagabandi et al, 2020 ).…”

Section: Dexterous Manipulation For Multi-fingered Robotic Hands With...mentioning

confidence: 99%

Dexterous Manipulation for Multi-Fingered Robotic Hands With Reinforcement Learning: A Review

Wang

2022

Front. Neurorobot.

View full text Add to dashboard Cite

With the increasing demand for the dexterity of robotic operation, dexterous manipulation of multi-fingered robotic hands with reinforcement learning is an interesting subject in the field of robotics research. Our purpose is to present a comprehensive review of the techniques for dexterous manipulation with multi-fingered robotic hands, such as the model-based approach without learning in early years, and the latest research and methodologies focused on the method based on reinforcement learning and its variations. This work attempts to summarize the evolution and the state of the art in this field and provide a summary of the current challenges and future directions in a way that allows future researchers to understand this field.

show abstract

“…Mouret & Clune (2015)). The reason or justification for this is not discussed, but it has seemingly been adopted in the wider MBRL setting (Shen et al, 2020;Omer et al, 2021;. However, offline RL is a totally different paradigm, where it is possible that access to compute is less of a bottleneck and it may be preferable to use more models to extract the most signal possible from the static dataset.…”

Section: How Many Models Do We Need?mentioning

confidence: 99%

Revisiting Design Choices in Offline Model-Based Reinforcement Learning

Liu¹,

Ball²,

Parker-Holder³

et al. 2021

Preprint

View full text Add to dashboard Cite

Offline reinforcement learning enables agents to leverage large pre-collected datasets of environment transitions to learn control policies, circumventing the need for potentially expensive or unsafe online data collection. Significant progress has been made recently in offline model-based reinforcement learning, approaches which leverage a learned dynamics model. This typically involves constructing a probabilistic model, and using the model uncertainty to penalize rewards where there is insufficient data, solving for a pessimistic MDP that lower bounds the true MDP. Existing methods, however, exhibit a breakdown between theory and practice, whereby pessimistic return ought to be bounded by the total variation distance of the model from the true dynamics, but is instead implemented through a penalty based on estimated model uncertainty. This has spawned a variety of uncertainty heuristics, with little to no comparison between differing approaches. In this paper, we compare these heuristics, and design novel protocols to investigate their interaction with other hyperparameters, such as the number of models, or imaginary rollout horizon. Using these insights, we show that selecting these key hyperparameters using Bayesian Optimization produces superior configurations that are vastly different to those currently used in existing hand-tuned state-of-the-art methods, and result in drastically stronger performance.

show abstract

Model Predictive-Actor Critic Reinforcement Learning for Dexterous Manipulation

Cited by 8 publications

References 3 publications

Review on human‐like robot manipulation using dexterous hands

Review on human‐like robot manipulation using dexterous hands

Dexterous Manipulation for Multi-Fingered Robotic Hands With Reinforcement Learning: A Review

Revisiting Design Choices in Offline Model-Based Reinforcement Learning

Contact Info

Product

Resources

About