Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning

Tan, Xiaoyu; Chng, Chin-Boon; Su, Ye; Lim, Kah Bin; Chui, Chee-Kong

doi:10.1109/lra.2019.2891311

Cited by 45 publications

(14 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…whereḟ ,f , ... f , f (4) and f (5) denote the first-order, secondorder, third-order, fourth-order and fifth-order derivatives, respectively; ξ 1 , ξ 2 , ξ 3 , ξ 4 , ξ 5 , ξ 6 and ξ 7 lie in the interval (kτ, (k + 1)τ ), ((k − 1)τ, kτ ), ((k − 2)τ, kτ ), ((k − 3)τ, kτ ), ((k −4)τ, kτ ), ((k −5)τ, kτ ), ((k −6)τ, kτ ) and ((k −7)τ, kτ ), correspondingly; symbol ! denotes the factorial operator.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Discrete-Time ZND Models Solving ALRMPC via Eight-Instant General and Other Formulas of ZeaD

Chen

Zhang

2019

IEEE Access

View full text Add to dashboard Cite

Repetitive motion planning and control (RMPC) of redundant robot manipulators is a fundamental and important problem widely existing in industrial manufacturing. In this paper, the accelerationlevel RMPC (ALRMPC) is studied and solved in a discrete-time manner. For solving this problem, a new ALRMPC scheme with feedback control term is derived and presented at first. Then, by adopting Lagrange's undetermined multipliers method and zeroing neural dynamics (ZND), a continuous-time ZND model, which is based on the new ALRMPC scheme, is developed and proposed. Besides, an eight-instant general formula with high precision is constructed, proposed and analyzed. By using this eight-instant general formula and other multiple-instant Zhang et al discretization (ZeaD) formulas to discretize the continuous-time ZND model, four discrete-time ZND (DTZND) models for solving ALRMPC are thus obtained. Finally, theoretical analyses and computer simulation experiment results further substantiate the effectiveness and accuracy of the proposed DTZND models.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Moreover, to eliminate the termsf (kτ ), ... f (kτ ) and f (4) Then, according to Routh stability criterion [33], [34], 7is finally obtained. The proof is thus completed.…”

Section: Discussionmentioning

confidence: 99%

Discrete-Time ZND Models Solving ALRMPC via Eight-Instant General and Other Formulas of ZeaD

Chen

Zhang

2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…At present, there are mainly two research directions, methods of value-based and policy-based. The representative algorithm based on value function is Q-learning in which the selection policy of action is greedy [ 54 ]. The algorithm updates the action-value function in accordance with the following formula:

…”

Section: Deep Reinforcement Learningmentioning

confidence: 99%

Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning

Jiang

Zeng

et al. 2021

Sensors

147

View full text Add to dashboard Cite

Dexterous manipulation of the robot is an important part of realizing intelligence, but manipulators can only perform simple tasks such as sorting and packing in a structured environment. In view of the existing problem, this paper presents a state-of-the-art survey on an intelligent robot with the capability of autonomous deciding and learning. The paper first reviews the main achievements and research of the robot, which were mainly based on the breakthrough of automatic control and hardware in mechanics. With the evolution of artificial intelligence, many pieces of research have made further progresses in adaptive and robust control. The survey reveals that the latest research in deep learning and reinforcement learning has paved the way for highly complex tasks to be performed by robots. Furthermore, deep reinforcement learning, imitation learning, and transfer learning in robot control are discussed in detail. Finally, major achievements based on these methods are summarized and analyzed thoroughly, and future research challenges are proposed.

show abstract

“…In [50], the Restricted Boltzmann machine with the hidden variables has been adopted to approximate the action and state of the RL, and the hidden variables can be estimated by state-action-reward-state-action (SARSA) algorithm. In [51], the HMM is used with RL to predict the zone in the 5G system. Similar work can be found in [52].…”

Section: Related Workmentioning

confidence: 99%

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Yang

Lin

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Applying model-based learning for the optimal decision of the multi-agent system is not trivial due to the expensive price or even the impossibility of obtaining the ground truth for training the model of the complex environment. Such as learning the optimal action of hydraulic supports in the top-coal caving, the optimal action could not accessible as the ground truth of the corresponding state in the intricate processes. Regarding the latent ground truth as the hidden variable is an effective method in the hidden Markov model. This paper extends the hidden variable of ground truth to the multi-agent system and proposes the hidden Markov random field (HMRF) model with reinforcement learning for optimizing the action decision of the multi-agent. In the HMRF model, the input states and the output actions of the multi-agent are considered as an observable random field and a latent Markov random field, respectively. Based on the HMRF model, the optimal decision is inferred by the maximum posterior probability with the prior probability obtained by Q-learning. Meanwhile, the parameters of the HMRF model are estimated by the expectation maximum algorithm. In the experiment, the top-coal caving demonstrates the effectiveness of the proposed method that the recall of top-coal is improved prominently with a very small price of increasing the rock-rate. Furthermore, the proposed method is employed to deal with the predator-preys problem in the gym. The experiment result shows that the communication between agents by the HMRF increases the reward of the preys. INDEX TERMS Hidden Markov random field, optimal decision, multi-agent, top-coal caving.

show abstract

Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning

Cited by 45 publications

References 14 publications

Discrete-Time ZND Models Solving ALRMPC via Eight-Instant General and Other Formulas of ZeaD

Discrete-Time ZND Models Solving ALRMPC via Eight-Instant General and Other Formulas of ZeaD

Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning

Hidden Markov Random Field for Multi-Agent Optimal Decision in Top-Coal Caving

Contact Info

Product

Resources

About