Gestures for Picture Archiving and Communication Systems (PACS) operation in the operating room: Is there any standard?

Abstract

ObjectiveGestural interfaces allow accessing and manipulating Electronic Medical Records (EMR) in hospitals while keeping a complete sterile environment. Particularly, in the Operating Room (OR), these interfaces enable surgeons to browse Picture Archiving and Communication System (PACS) without the need of delegating functions to the surgical staff. Existing gesture based medical interfaces rely on a suboptimal and an arbitrary small set of gestures that are mapped to a few commands available in PACS software. The objective of this work is to discuss a method to determine the most suitable set of gestures based on surgeon’s acceptability. To achieve this goal, the paper introduces two key innovations: (a) a novel methodology to incorporate gestures’ semantic properties into the agreement analysis, and (b) a new agreement metric to determine the most suitable gesture set for a PACS.Materials and methodsThree neurosurgical diagnostic tasks were conducted by nine neurosurgeons. The set of commands and gesture lexicons were determined using a Wizard of Oz paradigm. The gestures were decomposed into a set of 55 semantic properties based on the motion trajectory, orientation and pose of the surgeons’ hands and their ground truth values were manually annotated. Finally, a new agreement metric was developed, using the known Jaccard similarity to measure consensus between users over a gesture set.ResultsA set of 34 PACS commands were found to be a sufficient number of actions for PACS manipulation. In addition, it was found that there is a level of agreement of 0.29 among the surgeons over the gestures found. Two statistical tests including paired t-test and Mann Whitney Wilcoxon test were conducted between the proposed metric and the traditional agreement metric. It was found that the agreement values computed using the former metric are significantly higher (p < 0.001) for both tests.ConclusionsThis study reveals that the level of agreement among surgeons over the best gestures for PACS operation is higher than the previously reported metric (0.29 vs 0.13). This observation is based on the fact that the agreement focuses on main features of the gestures rather than the gestures themselves. The level of agreement is not very high, yet indicates a majority preference, and is better than using gestures based on authoritarian or arbitrary approaches. The methods described in this paper provide a guiding framework for the design of future gesture based PACS systems for the OR.

Agreement Study Using Gesture Description Analysis

“…In our previous work, we conducted a guessability study with a group of nine neurosurgeons to obtain their gestural preferences [15]. Each surgeon was asked to create gestures (proposals) for each of the 28 commands (referents) present in a radiology image browser.…”

“…Further, we utilized the gesture descriptors proposed in the literature [15], [31], [32] to create soft representations of the gestures. These descriptors were classified into three categories as shown in Table II: Motion -describes the movement of hands, Orientationthe mean direction in which the hand is pointing and State -number of fingers that are open [15]. By definition, the descriptors are binary and orthogonal i.e.…”

“…In our previous work [15], we proposed to incorporate gesture properties into agreement analysis. However, this approach was based on intuition rather than on a thorough mathematical grounding.…”

Making the invisible visible: New perspectives on the intersection of human-environment interactions of clinical teams in intensive care

“…There is increasing interest regarding the actions and gestures that healthcare workers use to interact with technology, and the cognitive processes that underpin these interactions. Research to date has predominantly focused on gesturebased control of image manipulation within medical scans, 1 particularly in operating theatre environments where the demands of sterility restrict the ability of practitioners to use touch-based interfaces. 2,3 However, many other healthcare environments require numerous human/technology interactions including the Neonatal Intensive Care Unit (NICU).…”

Gesture Agreement Assessment Using Description Vectors

“…Interestingly enough, Stern et al developed a very similar formula while attempting to quantify the agreement among user-elicited proposals [3]. Since their introduction, these metrics were widely adopted in several user elicitation studies to assess agreement [4], [15]- [18].…”