Domain adaptive Sim-to-Real segmentation of oropharyngeal organs

Wang, Guankun; Ren, Tian-Ao; Lai, Jianhuang; Bai, Linquan; Ren, Hongliang

doi:10.1007/s11517-023-02877-0

Cited by 4 publications

(1 citation statement)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that unless extensive data with various anatomic conditions are used, the reconstructed and phantom environments we present may not fully reflect the complexity and variety of real cases. The sim-to-real discrepancy at the current visibility class/grade can be further reduced by using Fourier domain adaptation [34] and style transfer [35] to even achieve pixel-grade cross-domain (SOFA/phantom) feature segmentation [36]. A modified oropharyngeal-tracheal 3D phantom [37] was directly imported into the scene in obj format.…”

Section: A Virtual Environmentmentioning

confidence: 99%

Sim-to-Real Transfer of Soft Robotic Navigation Strategies That Learns From the Virtual Eye-in-Hand Vision

Lai

Ren

Yue

et al. 2024

IEEE Trans. Ind. Inf.

Self Cite

View full text Add to dashboard Cite

To steer a soft robot precisely in an unconstructed environment with minimal collision remains an open challenge for soft robots. When the environments are unknown, prior motion planning for navigation may not always be available. This paper presents a novel Simto-Real method to guide a cable-driven soft robot in a static environment under the Simulation Open Framework Architecture (SOFA). The scenario aims to resemble one of the steps during a simplified transoral tracheal intubation process where a robotic endotracheal tube is guided to the upper trachea-larynx location by a flexible video-assisted endoscope/stylet. In SOFA, we employ the quadratic programming inverse solver to obtain collision-free motion strategies for the endoscope/stylet manipulation based on the robot model and encode the virtual eye-in-hand vision. Then, we associate the anatomical features recognized by the virtual vision and the joint space motion using a closedloop nonlinear autoregressive exogenous model (NARX) network. Afterward, we transfer the learned knowledge to the robot prototype, expecting it to navigate to the desired spot in a new phantom environment automatically based on its eye-in-hand vision only. Experiment results indicate that our soft robot can efficaciously navigate through the unstructured phantom to the desired spot with minimal collision motion according to what it has learned from the virtual environment. The results show that the average R-squared coefficient between the closed-loop NARXforecasted and SOFA-referenced robot's cable and prismatic joint space motion are 0.963 and 0.997, respectively. The eye-in-hand visions also demonstrate good alignment between the robot tip and the glottis.

show abstract

Section: A Virtual Environmentmentioning

confidence: 99%

Sim-to-Real Transfer of Soft Robotic Navigation Strategies That Learns From the Virtual Eye-in-Hand Vision

Lai

Ren

Yue

et al. 2024

IEEE Trans. Ind. Inf.

Self Cite

View full text Add to dashboard Cite

show abstract

Unsupervised Domain Adaptation for Simultaneous Segmentation and Classification of the Retinal Arteries and Veins

Xue,

Zhang,

et al. 2024

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Automatic segmentation of the fundus retinal vessels and accurate classification of the arterial and venous vessels play an important role in clinical diagnosis. This article proposes a fundus retinal vascular segmentation and arteriovenous classification network that combines the adversarial training and attention mechanism to address the issues of fundus retinal arteriovenous classification error and ambiguous segmentation of fine blood vessels. It consists of three core components: discriminator, generator, and segmenter. In order to address the domain shift issue, U‐Net is employed as a discriminator, and data samples for arterial and venous vessels are generated with a generator using an unsupervised domain adaption (UDA) approach. The classification of retinal arterial and venous vessels (A/V) as well as the segmentation of fine vessels is improved by adding a self‐attention mechanism to improve attention to vessel edge features and the terminal fine vessels. Non‐strided convolution and non‐pooled downsampling methods are also used to avoid losing fine‐grained information and learning less effective feature representations. The performance of multi‐class blood vessel segmentation is as follows, per test results on the DRIVE dataset: F1‐score (F1) has a value of 0.7496 and an accuracy of 0.9820. The accuracy of A/V categorization has increased by 1.35% when compared to AU‐Net. The outcomes demonstrate that by enhancing the baseline U‐Net, the strategy we suggested enhances the automated classification and segmentation of blood vessels.

show abstract

Sixty years in service to international biomedical engineering community

Magjarević

2023

Med Biol Eng Comput

View full text Add to dashboard Cite

Domain adaptive Sim-to-Real segmentation of oropharyngeal organs

Cited by 4 publications

References 31 publications

Sim-to-Real Transfer of Soft Robotic Navigation Strategies That Learns From the Virtual Eye-in-Hand Vision

Sim-to-Real Transfer of Soft Robotic Navigation Strategies That Learns From the Virtual Eye-in-Hand Vision

Unsupervised Domain Adaptation for Simultaneous Segmentation and Classification of the Retinal Arteries and Veins

Sixty years in service to international biomedical engineering community

Contact Info

Product

Resources

About