Eye-Gaze Driven Surgical Workflow Segmentation

James, Adam; Vieira, D.A.G.; Lo, Benny; Darzi, Ara; Yang, Guang‐Zhong

doi:10.1007/978-3-540-75759-7_14

Cited by 52 publications

(39 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, James et al [14] employed eye-gaze tracking for assessment and workflow recovery, with an emphasis on the detection of the clipping phase during LC. A neural network was used to perform the classification.…”

Section: Introductionmentioning

confidence: 99%

Surgical workflow analysis with Gaussian mixture multivariate autoregressive (GMMAR) models: a simulation study

Loukas

Georgiou

2013

Computer Aided Surgery

View full text Add to dashboard Cite

There is currently great interest in analyzing the workflow of minimally invasive operations performed in a physical or simulation setting, with the aim of extracting important information that can be used for skills improvement, optimization of intraoperative processes, and comparison of different interventional strategies. The first step in achieving this goal is to segment the operation into its key interventional phases, which is currently approached by modeling a multivariate signal that describes the temporal usage of a predefined set of tools. Although this technique has shown promising results, it is challenged by the manual extraction of the tool usage sequence and the inability to simultaneously evaluate the surgeon's skills. In this paper we describe an alternative methodology for surgical phase segmentation and performance analysis based on Gaussian mixture multivariate autoregressive (GMMAR) models of the hand kinematics. Unlike previous work in this area, our technique employs signals from orientation sensors, attached to the endoscopic instruments of a virtual reality simulator, without considering which tools are employed at each time-step of the operation. First, based on pre-segmented hand motion signals, a training set of regression coefficients is created for each surgical phase using multivariate autoregressive (MAR) models. Then, a signal from a new operation is processed with GMMAR, wherein each phase is modeled by a Gaussian component of regression coefficients. These coefficients are compared to those of the training set. The operation is segmented according to the prior probabilities of the surgical phases estimated via GMMAR. The method also allows for the study of motor behavior and hand motion synchronization demonstrated in each phase, a quality that can be incorporated into modern laparoscopic simulators for skills assessment.

show abstract

Section: Introductionmentioning

confidence: 99%

Surgical workflow analysis with Gaussian mixture multivariate autoregressive (GMMAR) models: a simulation study

Loukas

Georgiou

2013

Computer Aided Surgery

View full text Add to dashboard Cite

show abstract

“…Thus, data extraction is performed either from a human sight or from sensor devices. In this context, different methods have been recently used for data acquisition: patient specific procedures description [2][3][4], interview of the surgeons [5], sensor-based methods [6][7][8][9][10][11][12][13][14][15][16], using fixed protocols created by expert surgeons [17], or combination between them [18].…”

Section: Introductionmentioning

confidence: 99%

“…James et al [13] installed an eye-gaze tracking system on the surgeon combined with visual features to detect one important phase. Nara et al [14] introduced an ultrasonic location aware system that continuously tracks 3-D positions of the surgical staff.…”

Section: Introductionmentioning

confidence: 99%

Automatic Phases Recognition in Pituitary Surgeries by Microscope Images Classification

Lalys

Riffaud

Morandi

et al. 2010

Information Processing in Computer-Assisted Interventions

View full text Add to dashboard Cite

Abstract. The segmentation of the surgical workflow might be helpful for providing context-sensitive user interfaces, or generating automatic report. Our approach focused on the automatic recognition of surgical phases by microscope image classification. Our workflow, including images features extraction, image database labelisation, Principal Component Analysis (PCA) transformation and 10-fold cross-validation studies was performed on a specific type of neurosurgical intervention, the pituitary surgery. Six phases were defined by an expert for this type of intervention. We thus assessed machine learning algorithms along with the data dimension reduction. We finally kept 40 features from the PCA and found a best correct classification rate of the surgical phases of 82% with the multiclass Support Vector Machine.

show abstract

“…These four states were recognized using a combination of support vector machines and HMMs. Information obtained with an eye-tracker combined with visual features were used in [9] to recognize start and end of one surgical phase in a porcine laparoscopic cholecystectomy with an accuracy of 75%. In previous work, we already have segmented surgical phases of a laparoscopic cholecystectomy using a combination of Dynamic Time Warping (DTW) and AdaBoost [10].…”

Section: Introductionmentioning

confidence: 99%

Modeling and Online Recognition of Surgical Phases Using Hidden Markov Models

Blum

Padoy

Feußner

et al. 2008

Medical Image Computing and Computer-Assisted Intervention – MICCAI 2008

View full text Add to dashboard Cite

Abstract. The amount of signals that can be recorded during a surgery, like tracking data or state of instruments, is constantly growing. These signals can be used to better understand surgical workflow and to build surgical assist systems that are aware of the current state of a surgery. This is a crucial issue for designing future systems that provide contextsensitive information and user interfaces. In this paper, Hidden Markov Models (HMM) are used to model a laparoscopic cholecystectomy. Seventeen signals, representing tool usage, from twelve surgeries are used to train the model. The use of a model merging approach is proposed to build the HMM topology and compared to other methods of initializing a HMM. The merging method allows building a model at a very fine level of detail that also reveals the workflow of a surgery in a human-understandable way. Results for detecting the current phase of a surgery and for predicting the remaining time of the procedure are presented.

show abstract

Eye-Gaze Driven Surgical Workflow Segmentation

Cited by 52 publications

References 13 publications

Surgical workflow analysis with Gaussian mixture multivariate autoregressive (GMMAR) models: a simulation study

Surgical workflow analysis with Gaussian mixture multivariate autoregressive (GMMAR) models: a simulation study

Automatic Phases Recognition in Pituitary Surgeries by Microscope Images Classification

Modeling and Online Recognition of Surgical Phases Using Hidden Markov Models

Contact Info

Product

Resources

About