A low‐cost multimodal head‐mounted display system for neuroendoscopic surgery

Xu, Xinghua; Zheng, Yi; Yao, Shujing; Sun, Guochen; Xu, Bainan; Chen, Xiaolei

doi:10.1002/brb3.891

Cited by 8 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such devices include both the LMC [108,121,123,124] and MK [125]. In 2017, a portable low-cost simulator using the LMC [120] for basic motor skills learning in MIS was described, and so too were a simulator for endoscopic third ventriculostomy learning [66] and a head-mounted display system using Oculus Rift and the LMC to guide neuroendoscopic surgery by manipulating 3D images [70]. Others used the approach of tracking hand movements during MIS training [109,126].…”

Section: Resultsmentioning

confidence: 99%

“…Furthermore, 1 study compared 3 natural user interfaces (MK, the LMC, and the Myo armband) in combination with voice control to perform 2 hepatectomies and 2 partial nephrectomies on an experimental porcine model [52]; similar to the studies by Wright [66] and Xu [70], this study used 3D reconstructions of preoperative images of the patient, which were manipulated by gestures during surgery. However, the application of gesture control technology in these cases is not for training purposes but for surgical assistance and planification.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2019

J Med Internet Res

View full text Add to dashboard Cite

Background The increasingly pervasive presence of technology in the operating room raises the need to study the interaction between the surgeon and computer system. A new generation of tools known as commercial off-the-shelf (COTS) devices enabling touchless gesture–based human-computer interaction is currently being explored as a solution in surgical environments. Objective The aim of this systematic literature review was to provide an account of the state of the art of COTS devices in the detection of manual gestures in surgery and to identify their use as a simulation tool for motor skills teaching in minimally invasive surgery (MIS). Methods For this systematic literature review, a search was conducted in PubMed, Excerpta Medica dataBASE, ScienceDirect, Espacenet, OpenGrey, and the Institute of Electrical and Electronics Engineers databases. Articles published between January 2000 and December 2017 on the use of COTS devices for gesture detection in surgical environments and in simulation for surgical skills learning in MIS were evaluated and selected. Results A total of 3180 studies were identified, 86 of which met the search selection criteria. Microsoft Kinect (Microsoft Corp) and the Leap Motion Controller (Leap Motion Inc) were the most widely used COTS devices. The most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments for educational or interventional purposes. The possibility of using this technology to develop portable low-cost simulators for skills learning in MIS was also examined. As most of the articles identified in this systematic review were proof-of-concept or prototype user testing and feasibility testing studies, we concluded that the field was still in the exploratory phase in areas requiring touchless manipulation within environments and settings that must adhere to asepsis and antisepsis protocols, such as angiography suites and operating rooms. Conclusions COTS devices applied to hand and instrument gesture–based interfaces in the field of simulation for skills learning and training in MIS could open up a promising field to achieve ubiquitous training and presurgical warm up.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Alvarez-Lopez¹,

Maina²,

Saigí-Rubió³

2019

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…Devices for gesture-based human-computer interaction are a new way of interacting with virtual environments. This study’s simulator presents a new form of gesture-based simulation that is portable, low-cost, and enables ubiquitous learning and preoperative warm-up [ 14 , 127 , 128 ].…”

Section: Discussionmentioning

confidence: 99%

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

2020

View full text Add to dashboard Cite

Background Simulation in virtual environments has become a new paradigm for surgeon training in minimally invasive surgery (MIS). However, this technology is expensive and difficult to access. Objective This study aims first to describe the development of a new gesture-based simulator for learning skills in MIS and, second, to establish its fidelity to the criterion and sources of content-related validity evidence. Methods For the development of the gesture-mediated simulator for MIS using virtual reality (SIMISGEST-VR), a design-based research (DBR) paradigm was adopted. For the second objective, 30 participants completed a questionnaire, with responses scored on a 5-point Likert scale. A literature review on the validity of the MIS training-VR (MIST-VR) was conducted. The study of fidelity to the criterion was rated using a 10-item questionnaire, while the sources of content-related validity evidence were assessed using 10 questions about the simulator training capacity and 6 questions about MIS tasks, and an iterative process of instrument pilot testing was performed. Results A good enough prototype of a gesture-based simulator was developed with metrics and feedback for learning psychomotor skills in MIS. As per the survey conducted to assess the fidelity to the criterion, all 30 participants felt that most aspects of the simulator were adequately realistic and that it could be used as a tool for teaching basic psychomotor skills in laparoscopic surgery (Likert score: 4.07-4.73). The sources of content-related validity evidence showed that this study’s simulator is a reliable training tool and that the exercises enable learning of the basic psychomotor skills required in MIS (Likert score: 4.28-4.67). Conclusions The development of gesture-based 3D virtual environments for training and learning basic psychomotor skills in MIS opens up a new approach to low-cost, portable simulation that allows ubiquitous learning and preoperative warm-up. Fidelity to the criterion was duly evaluated, which allowed a good enough prototype to be achieved. Content-related validity evidence for SIMISGEST-VR was also obtained.

show abstract

“…Multimodality medical information fusion and processing have developed rapidly in the recent years. [ 12 ] This head-mounted display system allows the integration of preoperative radiological findings with monitoring of real-time endoscopic images and patient's vital signs. Using the head-mounted display system, the surgeon can view multiple images in one view using a split screen, regardless of the head position.…”

Section: Discussionmentioning

confidence: 99%

Surgical strategy of endoscopically assisted periradicular surgery using novel head-mounted display system

Sukegawa

Kanno

Matsuo

et al. 2020

Ann Maxillofac Surg

View full text Add to dashboard Cite

A novel head-mounted display offers high quality of endoscopic imagery in front of the eyes, irrespective of the head position. We present an application of the head-mounted display system as a personal integrated multi-image monitoring system in endoscopically assisted periradicular surgery. Our head-mounted display system displayed multiple forms of information as integrated, sharp, high-definition endoscope, biological monitor, and X-ray image (such as panoramic and computed tomography images) synchronously using a picture in picture. In addition, this system can cope with both the endoscopic field of view and the direct field of view. While monitoring the patient's general condition with a head-mounted display, the surgery was performed with endoscopic animation. We could also switch to the direct surgical field and endoscopic field of view smoothly without moving the head and without surgical interference. The availability of a head-mounted display system during endoscopically assisted periradicular surgery enabled the provision of a comfortable and appropriate surgical environment for the surgeon.

show abstract

A low‐cost multimodal head‐mounted display system for neuroendoscopic surgery

Cited by 8 publications

References 30 publications

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Use of Commercial Off-The-Shelf Devices for the Detection of Manual Gestures in Surgery: Systematic Literature Review

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

Surgical strategy of endoscopically assisted periradicular surgery using novel head-mounted display system

Contact Info

Product

Resources

About