Virtual pointer for gaze guidance in laparoscopic surgery

Feng, Yuanyuan; McGowan, Hannah; Semsar, Azin; Zahiri, H. Reza; George, Ivan; Park, Adrian; Kleinsmith, Andrea; Mentis, Helena M.

doi:10.1007/s00464-019-07141-x

Cited by 20 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From our prior work in surgical telementoring as well as other prior work in the field of CSCW, we knew that deictic referencing was an important part of collaboration (e.g. [3,6]). Thus, the team immediately agreed that we needed to find a mechanism for pointing and annotating a live mobile view as one would naturally do on a static video image.…”

Section: Example Collaborative Explorationmentioning

confidence: 99%

“…Collaborative systems that provide shared visual information through shared workspaces have been studied for decades in the field of computer supported cooperative work (CSCW), and their benefits when remote experts guide local novices during physical tasks are well known [5,7,8]. For example, shared workspaces improve situation awareness and are resources during conversational grounding [7] improving communication between collaborators [8], and virtual pointing in shared workspaces improves performance by reducing movement quantity for physical tasks [5] as implicit guidance becomes explicit visual cues [4].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AR HMD for Remote Instruction in Healthcare

Mentis

Avellino²,

Seo

2022

2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

View full text Add to dashboard Cite

In the following position paper we introduce the use of AR HMD for remote instruction in healthcare and present the challenges our team has faced in achieving this application in two contexts: surgical telementoring and paramedic teleconsulting. After the presentation of how these challenges come to be and indications on how to address them, we argue that those who wish to pursue this area of research must be grounded in best practices from the field of CSCW integrated with technical innovations in AR interaction development. This is a truly interdisciplinary research and development area that has many challenging topics to tackle through collaborative efforts.

show abstract

Section: Example Collaborative Explorationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

AR HMD for Remote Instruction in Healthcare

Mentis

Avellino²,

Seo

2022

2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

View full text Add to dashboard Cite

show abstract

“…Finally, Feng et al [15,16] presented a virtual pointer that was controlled by hand gestures and superimposed the laparoscopic video screen. This pointer was developed with the purpose of facilitating the adoption of professional vision.…”

Section: Related Workmentioning

confidence: 99%

HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training

et al. 2020

View full text Add to dashboard Cite

Purpose In laparoscopic surgery training, experts guide novice physicians to desired instrument positions or indicate relevant areas of interest. These instructions are usually given via verbal communication or using physical pointing devices. To facilitate a sterile work flow and to improve training, new guiding methods are needed. This work proposes to use optical see-through augmented reality to visualize an interactive virtual pointer on the laparoscopic. Methods After an interdisciplinary development, the pointer’s applicability and feasibility for training was evaluated and it was compared to a standard condition based on verbal and gestural communication only. In this study, ten surgical trainees were guided by an experienced trainer during cholecystectomies on a laparoscopic training simulator. All trainees completed a virtual cholecystectomy with and without the interactive virtual pointer in alternating order. Measures included procedure time, economy of movement and error rates. Results Results of standardized variables revealed significantly improved economy of movement (p = 0.047) and error rates (p = 0.047), as well as an overall improved user performance (Total z-score; p = 0.031) in conditions using the proposed method. Conclusion The proposed HoloPointer is a feasible and applicable tool for laparoscopic surgery training. It improved objective performance metrics without prolongation of the task completion time in this pre-clinical setup.

show abstract

“…In the field of visual attention guidance, many different methods have been proposed in recent years for application in images as well as videos; in non‐immersive settings [DMGB04, BMSG09, VMFS11, HKS16], in VR settings using Head‐Mounted Display (HMD) [LCH*17, GSEM17, GAM18, RAK18, GTA*19], and even in an immersive room‐scale projection system [GATM18, GTA*19]. Attention guidance as a support means may be helpful in various application scenarios, such as virtual training or remote teaching [dKJ17, FMS*19, YKB19b, YKB19a], guided exploration [LH19], multi‐monitor surveillance tasks [SKB19] or immersive story telling [SP19, SRD*19, LSGB20]. For our survey on related work, we will distinguish between passive and active methods, that is, they do or do not actively incorporate real‐time gaze tracking data.…”

Section: Related Workmentioning

confidence: 99%

Stereo Inverse Brightness Modulation for Guidance in Dynamic Panorama Videos in Virtual Reality

Grogorick

Tauscher

Heesen

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

The peak of virtual reality offers new exciting possibilities for the creation of media content but also poses new challenges. Some areas of interest might be overlooked because the visual content fills up a large portion of viewers' visual field. Moreover, this content is available in 360° around the viewer, yielding locations completely out of sight, making, for example, recall or storytelling in cinematic Virtual Reality (VR) quite difficult. In this paper, we present an evaluation of Stereo Inverse Brightness Modulation for effective and subtle guidance of participants' attention while navigating dynamic virtual environments. The used technique exploits the binocular rivalry effect from human stereo vision and was previously shown to be effective in static environments. Moreover, we propose an extension of the method for successful guidance towards target locations outside the initial visual field. We conduct three perceptual studies, using 13 distinct panorama videos and two VR systems (a VR head mounted display and a fully immersive dome projection system), to investigate (1) general applicability to dynamic environments, (2) stimulus parameter and VR system influence, and (3) effectiveness of the proposed extension for out‐of‐sight targets. Our results prove the applicability of the method to dynamic environments while maintaining its unobtrusive appearance.

show abstract

Virtual pointer for gaze guidance in laparoscopic surgery

Cited by 20 publications

References 22 publications

AR HMD for Remote Instruction in Healthcare

AR HMD for Remote Instruction in Healthcare

HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training

Stereo Inverse Brightness Modulation for Guidance in Dynamic Panorama Videos in Virtual Reality

Contact Info

Product

Resources

About