Application of Mixed Reality Technology in Visualization of Medical Operations

Gao, Yuan; Tan, Ke; Sun, Jian; Jiang, Tao; Zou, Xiaowen

doi:10.24920/003564

Cited by 14 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The visualization systems for PM ought to be highly interactive, immersive, and create a kind of reality that allows multiple analysts to manipulate/zoom into the virtual digital object or system under examination for improved analytical results. The use of the VR [19], [127], AR, [128], [129], MR [130], and holographic display [131], [132] technologies are increasingly gaining attention in medical practice. For instance, specialists can examine the virtual anatomy or the digital model of a patient that is thousands of miles away on an XR or 3D holographic display and make some changes to the models as a pre-therapeutic procedure.…”

Section: F Advanced Visualization Platformsmentioning

confidence: 99%

Enabling Precision Medicine via Contemporary and Future Communication Technologies: A Survey

2023

View full text Add to dashboard Cite

Precision medicine (PM) is an innovative medical approach that considers differences in the individuals' omics, medical histories, lifestyles, and environmental information in treating diseases. To fully achieve the envisaged gains of PM, various contemporary and future technologies have to be employed, among which are nanotechnology, sensor network, big data, and artificial intelligence. These technologies and other applications require a communication network that will enable them to work in tandem for the benefit of PM. Hence, communication technology serves as the nervous system of PM, without which the entire system collapses. Therefore, it is essential to explore and determine the candidate communication technology requirements that can guarantee the envisioned gains of PM. To the best of our knowledge, no work exploring how communication technology directly impacts the development and deployment of PM solutions exists. This survey paper is designed to stimulate discussions on PM from the communication engineering perspective. We introduce the fundamentals of PM and the demands in terms of quality of service that each of the enabling technologies of PM places on the communication network. We explore the information in the literature to suggest the ideal metric values of the key performance indicators for the implementation of the different components of PM. The comparative analysis of the suitability of the contemporary and future communication technologies for PM implementation is discussed. Finally, some open research challenges for the candidate communication technologies that will enable the full implementation of PM solutions are highlighted.INDEX TERMS Precision medicine, fourth industrial revolution (4IR) technologies, communication technologies, 5G, 6G.

show abstract

Section: F Advanced Visualization Platformsmentioning

confidence: 99%

Enabling Precision Medicine via Contemporary and Future Communication Technologies: A Survey

2023

View full text Add to dashboard Cite

show abstract

“…Anglo Croatti used digital twinning technology to realize the application of trauma management. This perspective makes it possible to explore the use of a DT's simulation features on the agent side to support agent decision-making [43]. In the course of cancer treatment, late diagnosis is a leading cause of medical complications.…”

Section: Digital Twin Implementations In Medical Simulationmentioning

confidence: 99%

Digital-Twin-Enabled IoMT System for Surgical Simulation Using rAC-GAN

Tai

Zhang

et al. 2022

IEEE Internet Things J.

View full text Add to dashboard Cite

A digital twin-enabled Internet of Medical Things (IoMT) system for telemedical simulation is developed, systematically integrated with mixed reality (MR), 5G cloud computing, and a generative adversarial network (GAN) to achieve remote lung cancer implementation. Patient-specific data from 90 lung cancer with pulmonary embolism (PE)-positive patients, with 1372 lung cancer control groups, were gathered from Qujing and Dehong, and then transmitted and preprocessed using 5G. A novel robust auxiliary classifier generative adversarial network (rAC-GAN)-based intelligent network is employed to facilitate lung cancer with the PE prediction model. To improve the accuracy and immersion during remote surgical implementation, a real-time operating room perspective from the perception layer with a surgical navigation image is projected to the surgeon's helmet in the application layer using the digital twin-based MR guide clue with 5G. The accuracies of the area under the curve (AUC) of our new intelligent IoMT system were 0.92, and 0.93. Furthermore, the pathogenic features learned from our rAC-GAN model are highly consistent with the statistical epidemiological results. The proposed intelligent IoMT system generates significant performance improvement to process substantial clinical data at cloud centers and shows a novel framework for remote medical data transfer and deep learning analytics for digital twin-based surgical implementation.

show abstract

“…Video-feedback systems, such as those implemented through mixed reality (MR), require a lot of bandwidth and have also been shown to be a major source of delay [16]- [27]. Even with significant bandwidth allocation, online VR connections may involve delays of up to 100 ms, particularly during extreme events, such as those likely to occur in the implemented environment of the proposed system [28,29].…”

Section: Introductionmentioning

confidence: 99%

Latency mitigation using applied HMMs for mixed reality-enhanced intuitive teleoperation in intelligent robotic welding

Lloyd

Chen

et al. 2023

Int J Adv Manuf Technol

View full text Add to dashboard Cite

Latency mitigation is crucial to increasing operational success, ease of use, and product quality in telemanipulation tasks when remotely guiding complex robotic systems. Hardware limitations have created a gap in performance optimization due to large teleoperation delays, which machine learning techniques could fill with lower time, improved performance, and reduced operating costs. Hidden Markov models (HMMs), in particular, have been explored to alleviate the issue due to their relative ease of use. A mixed reality-enhanced intuitive teleoperation framework for immersive and intuitive telerobotic welding is presented. The proposed system implements an HMM generative algorithm to learn and predict human-welder motion to enable a low-cost solution, combining smoothing and forecasting techniques to minimize robotic teleoperation time delay. The predicted welding motion system is simple to implement, can be used as a general solution to solve time delays, and is accurate. More specifically, it provides a 66% RMSE reduction compared to the application without HMM, which may be further optimized by up to 38%. Experiments show the HMM generative algorithm lets humans conduct tele-robot-assisted welding with better performance.

show abstract

Application of Mixed Reality Technology in Visualization of Medical Operations

Cited by 14 publications

References 15 publications

Enabling Precision Medicine via Contemporary and Future Communication Technologies: A Survey

Enabling Precision Medicine via Contemporary and Future Communication Technologies: A Survey

Digital-Twin-Enabled IoMT System for Surgical Simulation Using rAC-GAN

Latency mitigation using applied HMMs for mixed reality-enhanced intuitive teleoperation in intelligent robotic welding

Contact Info

Product

Resources

About