Sonification as a reliable alternative to conventional visual surgical navigation

Abstract

Despite the undeniable advantages of image-guided surgical assistance systems in terms of accuracy, such systems have not yet fully met surgeons’ needs or expectations regarding usability, time efficiency, and their integration into the surgical workflow. On the other hand, perceptual studies have shown that presenting independent but causally correlated information via multimodal feedback involving different sensory modalities can improve task performance. This article investigates an alternative method for computer-assisted surgical navigation, introduces a novel four-DOF sonification methodology for navigated pedicle screw placement, and discusses advanced solutions based on multisensory feedback. The proposed method comprises a novel four-DOF sonification solution for alignment tasks in four degrees of freedom based on frequency modulation synthesis. We compared the resulting accuracy and execution time of the proposed sonification method with visual navigation, which is currently considered the state of the art. We conducted a phantom study in which 17 surgeons executed the pedicle screw placement task in the lumbar spine, guided by either the proposed sonification-based or the traditional visual navigation method. The results demonstrated that the proposed method is as accurate as the state of the art while decreasing the surgeon’s need to focus on visual navigation displays instead of the natural focus on surgical tools and targeted anatomy during task execution.

Comparing Performance and Preference of Visually Impaired Individuals in Object Localization: Tactile, Verbal, and Sonification Cueing Modalities

“…Also, the continuous beeping might be annoying, and the cognitive workload required to convert the beeping sound into spatial information might contribute to the observed delay, as suggested by [20]. While target sonification might prove advantageous for sighted individuals, particularly in scenarios involving intricate visual guidance such as surgery [21], for the visually impaired population, sonification may present challenges due to the inherent need for comprehensive auditory cues and efficient cognitive processing. Hence, pitch sonification is the least advisable cueing modality for object localization among the three assessed in this study.…”

Ultra-responsive, low-dimensional unfamiliar movement sonification guides unconstrained reaches to invisible targets in 3D space

“…Similar sonifications demonstrated to be effective have been limited to low DoF tasks (Ghai et al 2018). A study demonstrating that sonified spatial information is used in a high DoF movement (drill alignment in phantom spine surgery) decomposed the high DoF movement into two low DoF stages (2 DoF each) and used a different low-dimensional (2D) sonification in each stage (Matinfar et al 2023).…”

“…This preference derives from ecological approaches to auditory perception (Gaver 1993) and in the sonification literature is motivated by the idea that it's easier to interpret feedback that exploits known associations. However, many studies do employ unfamiliar sonifications, in the form of "pure" tones or modulations of white noise (Effenberg et al 2016;Boyer et al 2017;Ghai et al 2018;Connor et al 2022;Linnhoff et al 2022;Matinfar et al 2023).…”

“…However, we expected relatively fast learning, given the responsiveness and lowdimensionality of the system. For example, Matinfar et al (2023) gave participants only 10 trials of training on their unfamiliar, low-dimensional sonification. So, we divided the 50 feedback reaches (reaches 26-75) into a 25-reach learning period (reaches 26-50) and final 25 (reaches 51-75), as shown in Fig 3B . An informal visual inspection of mean error by trial verified that, on average, participant performance quickly improved (error dropped) in the 25-reach learning period and plateaued before the final 25 reaches.…”

Evaluation of the Effectiveness of Sonification for Time-series Data Exploration

“…Our ability to process multiple sound streams and recognize patterns (Bregman 1990;Hermann et al 2011;Sawe et al 2020), decode melodic contours, and notice slight variations in frequency, enhances the potential to uncover significant patterns embedded in the data (Kramer et al 1999). These qualities make sonification particularly interesting for conveying time-dependent information, such as those collected through monitoring (e.g., see the sonification of electroencephalograms of epileptic seizures; Supper 2012; Matinfar et al 2023). Traditionally, astronomy was studied and disseminated through visual means, such as images, animations, and graphs.…”