Modeling and Online Recognition of Surgical Phases Using Hidden Markov Models

Blum, Tobias; Padoy, Nicolas; Feußner, H.; Navab, Nassir

doi:10.1007/978-3-540-85990-1_75

Cited by 37 publications

(36 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this sense HMMs are key techniques, proven both by their recurrence and results. Their usefulness for surgery modeling has been exploited successfully in other surgical fields such as robotics [116,117]. The differences with simple MM in terms of results are not so significant, though their nature makes HMMs more flexible to the requirements of competence assessment.…”

Section: Table6mentioning

confidence: 99%

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

García

Sánchez-González

Lamata

et al. 2011

Journal of Surgical Research

133

View full text Add to dashboard Cite

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentortrainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons' psychomotor skills. Ongoing development of tracking systems and software Solutions has allowed for the expansión of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentáis of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as 1 To whom correspondence and reprint requests should be addressed at Bioengineering and Telemedicine Centre (GBT), ETSI Telecomunicación, Universidad Politécnica de Madrid (UPM). Avda Complutense, 30, 28040, Madrid, Spain. E-mail: ioropesa@gbt.tfo. upm.es. well as optimize their already overloaded training schedules.

show abstract

Section: Table6mentioning

confidence: 99%

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

García

Sánchez-González

Lamata

et al. 2011

Journal of Surgical Research

133

View full text Add to dashboard Cite

show abstract

“…Bouarfa et al [11] have presented a Markov-based approach for inferring high-level tasks from a set of low-level signals of instrument usage extracted from video recordings of the tool tray view, with five surgical steps of laparoscopic cholecystectomy being segmented using a vector of 10 signals. Padoy et al [12] proposed a left-to-right HMM based on data containing visual cues computed from the endoscopic video, and this same group has proposed several variants of the HMM algorithm for analyzing the workflow of laparoscopic cholecystectomy, such as the use of a model merging approach to build the HMM topology [13] and, more recently, a generalized framework that addresses data where the phases have been partially labeled [3].…”

Section: Introductionmentioning

confidence: 99%

Surgical workflow analysis with Gaussian mixture multivariate autoregressive (GMMAR) models: a simulation study

Loukas

Georgiou

2013

Computer Aided Surgery

View full text Add to dashboard Cite

There is currently great interest in analyzing the workflow of minimally invasive operations performed in a physical or simulation setting, with the aim of extracting important information that can be used for skills improvement, optimization of intraoperative processes, and comparison of different interventional strategies. The first step in achieving this goal is to segment the operation into its key interventional phases, which is currently approached by modeling a multivariate signal that describes the temporal usage of a predefined set of tools. Although this technique has shown promising results, it is challenged by the manual extraction of the tool usage sequence and the inability to simultaneously evaluate the surgeon's skills. In this paper we describe an alternative methodology for surgical phase segmentation and performance analysis based on Gaussian mixture multivariate autoregressive (GMMAR) models of the hand kinematics. Unlike previous work in this area, our technique employs signals from orientation sensors, attached to the endoscopic instruments of a virtual reality simulator, without considering which tools are employed at each time-step of the operation. First, based on pre-segmented hand motion signals, a training set of regression coefficients is created for each surgical phase using multivariate autoregressive (MAR) models. Then, a signal from a new operation is processed with GMMAR, wherein each phase is modeled by a Gaussian component of regression coefficients. These coefficients are compared to those of the training set. The operation is segmented according to the prior probabilities of the surgical phases estimated via GMMAR. The method also allows for the study of motor behavior and hand motion synchronization demonstrated in each phase, a quality that can be incorporated into modern laparoscopic simulators for skills assessment.

show abstract

“…To solve this problem, an analysis of endoscopic and microscopic video has been proposed [31][32][33]. Markov models and dynamic time warping (DTW) were used to identify single worksteps based on the presence of surgical instruments [34][35][36][37], and radiofrequency identification (RFID), visual approaches, and weight analysis methods [38] have been employed to recognize instrument use [37,39,40]. For laparoscopy, another approach using a camera on the trocar and color wheels was proposed by Toti et al [41].…”

Section: Data Acquisition By Sensor Systemsmentioning

confidence: 99%

“…All of these approaches address different levels of granularity from ranged gesture recognition (surgemes) [42,44,45] to low-level tasks [33], high-level tasks [46,47], and intervention phases [35,48]. James et al [49] tried to recognize the current surgical situation indirectly by estimating the positions and movements of the members of surgical teams within the OR or by deriving information from other indirect features.…”

Section: Data Acquisition By Sensor Systemsmentioning

confidence: 99%

Surgical process modeling

Neumuth

2017

Innovative Surgical Sciences

View full text Add to dashboard Cite

Due to the rapidly evolving medical, technological, and technical possibilities, surgical procedures are becoming more and more complex. On the one hand, this offers an increasing number of advantages for patients, such as enhanced patient safety, minimal invasive interventions, and less medical malpractices. On the other hand, it also heightens pressure on surgeons and other clinical staff and has brought about a new policy in hospitals, which must rely on a great number of economic, social, psychological, qualitative, practical, and technological resources. As a result, medical disciplines, such as surgery, are slowly merging with technical disciplines. However, this synergy is not yet fully matured. The current information and communication technology in hospitals cannot manage the clinical and operational sequence adequately. The consequences are breaches in the surgical workflow, extensions in procedure times, and media disruptions. Furthermore, the data accrued in operating rooms (ORs) by surgeons and systems are not sufficiently implemented. A flood of information, "big data", is available from information systems. That might be deployed in the context of Medicine 4.0 to facilitate the surgical treatment. However, it is unused due to infrastructure breaches or communication errors. Surgical process models (SPMs) alleviate these problems. They can be defined as simplified, formal, or semiformal representations of a network of surgery-related activities, reflecting a predefined subset of interest. They can employ different means of generation, languages, and data acquisition strategies. They can represent surgical interventions with high resolution, offering qualifiable and quantifiable information on the course of the intervention on the level of single, minute, surgical work-steps. The basic idea is to gather information concerning the surgical intervention and its activities, such as performance time, surgical instrument used, trajectories, movements, or intervention phases. These data can be gathered by means of workflow recordings.These recordings are abstracted to represent an individual surgical process as a model and are an essential requirement to enable Medicine 4.0 in the OR. Further abstraction can be generated by merging individual process models to form generic SPMs to increase the validity for a larger number of patients. Furthermore, these models can be applied in a wide variety of use-cases. In this regard, the term "modeling" can be used to support either one or more of the following tasks: "to describe", "to understand", "to explain", to optimize", "to learn", "to teach", or "to automate". Possible use-cases are requirements analyses, evaluating surgical assist systems, generating surgeon-specific training-recommendation, creating workflow management systems for ORs, and comparing different surgical strategies. The presented chapter will give an introduction into this challenging topic, presenting different methods to generate SPMs from the workflow in the OR, as well as various...

show abstract

Modeling and Online Recognition of Surgical Phases Using Hidden Markov Models

Cited by 37 publications

References 10 publications

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

Surgical workflow analysis with Gaussian mixture multivariate autoregressive (GMMAR) models: a simulation study

Surgical process modeling

Contact Info

Product

Resources

About