Evaluation of Gesture Based Interfaces for Medical Volume Visualization Tasks

Abstract

Interactive systems are increasingly used in medical applications with the widespread availability of various imaging modalities. Gesture-based interfaces can be beneficial to interact with these kinds of systems in a variety of settings, as they can be easier to learn and can eliminate several shortcomings of traditional tactile systems, especially for surgical applications. We conducted two user studies that explore different gesture-based interfaces for interaction with volume visualizations. The first experiment focused on rotation tasks, where the performance of the gesture-based interface (using Microsoft Kinect) was compared to using the mouse. The second experiment studied localization of internal structures, comparing slice-based visualizations via gestures and the mouse, in addition to a 3D Magic Lens visualization. The results of the user studies showed that the gesture-based interface outperform the traditional mouse both in time and accuracy in the orientation matching task. The traditional mouse was the superior interface for the second experiment in terms of accuracy. However, the gesture-based Magic Lens interface was found to have the fastest target localization time. We discuss these findings and their further implications in the use of gesture-based interfaces in medical volume visualization, and discuss the possible underlying psychological mechanisms why these methods can outperform traditional interaction methods

Interaction for Immersive Analytics

“…Users wearing datagloves can rotate the visualization to inspect it from all sides and select nodes using hand gestures. Kirmizibayrak et al [87] used gestures to control a visualization of medical volume data. Their studies show that gestures can outperform mouse interaction in rotation tasks.…”

Hand Gesture Recognition in Real Time for Automotive Interfaces: A Multimodal Vision-Based Approach and Evaluations

“…Hand Gesture Recognition With RGBD Cues: The introduction of high-quality depth sensors at a lower cost, such as the Microsoft Kinect, facilitated the development of many gesture recognition systems. In particular, hand gesture recognition systems were developed with applications in the fields of sign language recognition [31]- [34], driver assistance [35], [36], smart environments [21], [37], [38], video games [39], medical instrumentation [40], [41], and other human-computer interfaces [42]- [44]. Hand gesture recognition systems commonly use depth information for background removal purposes [45]- [48].…”

Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images

“…Other relevant prior work 13 required the use of voice commands to switch between modes allowing the tracked movement of the hands to manipulate images (similar to previous studies [21][22][23] where the position of the hands is used like a mouse pointer). Alternatively, our system recognizes the movement of the hands to manipulate images.…”