Intraoperative multichannel audio-visual information recording and automatic surgical phase and incident detection

Abstract

Identification, analysis, and treatment of potential risk in surgical workflow are the key to decrease medical errors in operating room. For the automatic analysis of recorded surgical information, this study reports multichannel audio visual recording system, and its review and analysis system. Motion in operating room is quantified using video file size without motion tracking. Conversation among surgical staff is quantified using fast Fourier transformation and frequency filter without speech recognition. The results suggested the progression phase of surgical procedure.

LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition

“…Blum et al [14] represent surgeries with snippets of signals. Suzuki et al [15] quantify motion using video file size and oral conversations in the OR, without explicit speech recognition for surgical phase detection. Both approaches fail to include medical background knowledge in a standardized, reusable way.…”

Untitled

“…Several cameras were mounted in the operating room to shoot the motion of surgical staff [11]. Figure 3(a) shows four cameras mounted on the surgical light.…”

“…Some of the streams are recorded on computers through USB-connected video capture devices, and some are broadcast to a doctor's office and the research institute using a networked video server via the intra-hospital network. In this study, six channels of video were recorded simultaneously using a developed system [11]. Several channels of video are integrated using a video mixing device (TAHOMA LE-16 multiviewer, Apantac LLC, USA), and shown on the in-room monitor (CELL REGZA 55X2, Toshiba, Japan).…”

“…The identification of incidents by a medical doctor is timeconsuming, and artificial intelligence is insufficient for surgical workflow analysis. This study thus uses the hypothesis that the movement of surgical staff increases when an incident occurs in order to compensate for the problem or when the surgical phase changes [11]. If the motion of surgical staff can be measured in a video and time periods with a relatively large amount of motion can be found, the time periods can represent candidate events that contain medical errors.…”

Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy

“…[8]. Similarly, Suzuki et al [9] use changes in encoding efficiency of videos to quantify movement for surgical phase detection. To further abstract from raw signals, there are attempts to extract features for a more concise representation of surgical workflows.…”