Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force

Yong-qing, FU; Lin, Weiyang; Yu, Xinghu; Rodríguez-Andina, Juan J.; Gao, Huijun

doi:10.1109/tie.2022.3201322

Cited by 18 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A compatible approximator A π ψ w (s,a) ≈ ∇ ψ log π ψ (a|s) T w [24] that is suitable for stochastic policies and produces unbiased estimates of the gradient is used in the eNAC algorithm [19], simplifying the gradient calculation to Equations ( 7) and (8).…”

Section: Enac Methodsmentioning

confidence: 99%

“…Although advances in the modeling and control of robot motors have made the position control of robot endeffector more accurate [5,6], the contact forces are also affected by the environment. The uncontrollable contact force may cause damage to the robot or fragile touched objects such as mirrors [7] or human bodies [8]. In addition to safety, contact force control, such as force tracking, has a wide range of applications, especially when the visual serving is inaccessible or the position of the object being touched is biased towards contact operations such as grasping, manufacturing, assembly and tool wear [9][10][11][12].…”

Section: Introduction 1motivationsmentioning

confidence: 99%

See 1 more Smart Citation

Robot Manipulation Skills Transfer for Sim-to-Real in Unstructured Environments

Yin

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

Robot force control that needs to be customized for the robot structure in unstructured environments with difficult-to-tune parameters guarantees robots’ compliance and safe human–robot interaction in an increasingly expanding work environment. Although reinforcement learning provides a new idea for the adaptive adjustment of these parameters, the policy often needs to be trained from scratch when used in new robotics, even in the same task. This paper proposes the episodic Natural Actor-Critic algorithm with action limits to improve robot admittance control and transfer motor skills between robots. The motion skills learned by simple simulated robots can be applied to complex real robots, reducing the difficulty of training and time consumption. The admittance control ensures the realizability and mobility of the robot’s compliance in all directions. At the same time, the reinforcement learning algorithm builds up the environment model and realizes the adaptive adjustment of the impedance parameters during the robot’s movement. In typical robot contact tasks, motor skills are trained in a robot with a simple structure in simulation and used for a robot with a complex structure in reality to perform the same task. The real robot’s performance in each task is similar to the simulated robot’s in the same environment, which verifies the method’s effectiveness.

show abstract

Section: Enac Methodsmentioning

confidence: 99%

Section: Introduction 1motivationsmentioning

confidence: 99%

Robot Manipulation Skills Transfer for Sim-to-Real in Unstructured Environments

Yin

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, some advanced teleoperation with multiple-modal interface has been studied. In [23], an Augmented reality (AR) teleoperation interface was developed to reduce the time delay in ultrasound scanning. In [33], a multimodal feedback teleoperation system was developed to provide sonographers with a better experience by designing a virtual ultrasound probe on the human side.…”

Section: A Teleoperation-based Robot-assisted Sonographymentioning

confidence: 99%

“…• We develop a teleoperation system for robot-assisted sonography with two different interfaces, a haptic devicebased interface and a low-cost 3D Mouse-based interface, which can achieve continuous and intuitive telemanipulation by a leader device with a small workspace, compared with [23], [24], and [28]. We evaluate these two teleoperation interfaces, and guidance on designing teleoperation interface for tele-sonography is provided.…”

Section: Introductionmentioning

confidence: 99%

Design and Quantitative Assessment of Teleoperation-Based Human–Robot Collaboration Method for Robot-Assisted Sonography

Si,

Wang,

Yang

2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

Tele-echography has emerged as a promising and effective solution, leveraging the expertise of sonographers and the autonomy of robots to perform ultrasound scanning for patients residing in remote areas, without the need for in-person visits by the sonographer. Designing effective and natural human-robot interfaces for tele-echography remains challenging, with patient safety being a critical concern. In this article, we develop a teleoperation system for robot-assisted sonography with two different interfaces, a haptic device-based interface and a low-cost 3D Mouse-based interface, which can achieve continuous and intuitive telemanipulation by a leader device with a small workspace. To achieve compliant interaction with patients, we design impedance controllers in Cartesian space to track the desired position and orientation for these two teleoperation interfaces. We also propose comprehensive evaluation metrics of robot-assisted sonography, including subjective and objective evaluation, to evaluate tele-echography interfaces and control performance. We evaluate the ergonomic performance based on the estimated muscle fatigue and the acquired ultrasound image quality. We conduct user studies based on the NASA Task Load Index to evaluate the performance of these two humanrobot interfaces. The tracking performance and the quantitative comparison of these two teleoperation interfaces are conducted by the Franka Emika Panda robot. The results and findings provide guidance on human-robot collaboration design and implementation for robot-assisted sonography.Note to Practitioners-Robot-assisted sonography has demonstrated efficacy in medical diagnosis during clinical trials. However, deploying fully autonomous robots for ultrasound scanning remains challenging due to various constraints in practice, such as patient safety, dynamic tasks, and environmental uncertainties. Semi-autonomous or teleoperation-based robot sonography Manuscript

show abstract

“…Compared to tactile feedback devices, current kinesthetic feedback devices provide feedback through position, velocity, and torque information from the body's neighboring parts and have the advantages of multiple degrees of freedom (DoF) and a wide dynamic range. Fu Yongqing et al [19] presented a new method to implement augmented reality (AR) and kinesthetic feedback for remote operation of ultrasound systems to achieve real-time feedback. Giuk Lee et al [20] designed a column-type force-feedback device in which the entire device was mounted on a column that can move up and down, capable of providing a larger interaction space.…”

Section: Introductionmentioning

confidence: 99%

Design and Research of Multimodal Fusion Feedback Device Based on Virtual Interactive System

Zhang

Shi

et al. 2023

Actuators

View full text Add to dashboard Cite

This paper proposes a kinesthetic–tactile fusion feedback system based on virtual interaction. Combining the results of human fingertip deformation characteristics analysis and an upper limb motion mechanism, a fingertip tactile feedback device and an arm kinesthetic feedback device are designed and analyzed for blind instructors. In order to verify the effectiveness of the method, virtual touch experiments are established through the mapping relationship between the master–slave and virtual end. The results showed that the average recognition rate of virtual objects is 79.58%, and the recognition speed is improved by 41.9% compared with the one without force feedback, indicating that the kinesthetic–tactile feedback device can provide more haptic perception information in virtual feedback and improve the recognition rate of haptic perception.

show abstract

Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force

Cited by 18 publications

References 20 publications

Robot Manipulation Skills Transfer for Sim-to-Real in Unstructured Environments

Robot Manipulation Skills Transfer for Sim-to-Real in Unstructured Environments

Design and Quantitative Assessment of Teleoperation-Based Human–Robot Collaboration Method for Robot-Assisted Sonography

Design and Research of Multimodal Fusion Feedback Device Based on Virtual Interactive System

Contact Info

Product

Resources

About