Purpose: Automated delineation of structures and organs is a key step in medical imaging. However, due to the large number and diversity of structures and the large variety of segmentation algorithms, a consensus is lacking as to which automated segmentation method works best for certain applications. Segmentation challenges are a good approach for unbiased evaluation and comparison of segmentation algorithms. Methods: In this work, we describe and present the results of the Head and Neck Auto-Segmentation Challenge 2015, a satellite event at the Medical Image Computing and Computer Assisted Interventions (MICCAI) 2015 conference. Six teams participated in a challenge to segment nine structures in the head and neck region of CT images: brainstem, mandible, chiasm, bilateral optic nerves, bilateral parotid glands, and bilateral submandibular glands. Results: This paper presents the quantitative results of this challenge using multiple established error metrics and a well-defined ranking system. The strengths and weaknesses of the different auto-segmentation approaches are analyzed and discussed. Conclusions: The Head and Neck Auto-Segmentation Challenge 2015 was a good opportunity to assess the current state-of-the-art in segmentation of organs at risk for radiotherapy treatment. Participating teams had the possibility to compare their approaches to other methods under unbiased and standardized circumstances. The results demonstrate a clear tendency toward more general purpose and fewer structure-specific segmentation algorithms.

Background Objective assessment of surgical skills is resource intensive and requires valuable time of expert surgeons. The goal of this study was to assess the ability of a large group of laypersons using a crowd-sourcing tool to grade a surgical procedure (cricothyrotomy) performed on a simulator. The grading included an assessment of the entire procedure by completing an objective assessment of technical skills survey. Materials and methods Two groups of graders were recruited as follows: (1) Amazon Mechanical Turk users and (2) three expert surgeons from University of Washington Department of Otolaryngology. Graders were presented with a video of participants performing the procedure on the simulator and were asked to grade the video using the objective assessment of technical skills questions. Mechanical Turk users were paid $0.50 for each completed survey. It took 10 h to obtain all responses from 30 Mechanical Turk users for 26 training participants (26 videos/tasks), whereas it took 60 d for three expert surgeons to complete the same 26 tasks. Results The assessment of surgical performance by a group (n = 30) of laypersons matched the assessment by a group (n = 3) of expert surgeons with a good level of agreement determined by Cronbach alpha coefficient = 0.83. Conclusions We found crowd sourcing was an efficient, accurate, and inexpensive method for skills assessment with a good level of agreement to experts’ grading.

We present a fully automatic method for segmenting orbital structures (globes, optic nerves, and extraocular muscles) in CT images. Prior anatomical knowledge, such as shape, intensity, and spatial relationships of organs and landmarks, were utilized to define a volume of interest (VOI) that contains the desired structures. Then, VOI was used for fast localization and successful segmentation of each structure using predefined rules. Testing our method with 30 publicly available datasets, the average Dice similarity coefficient for right and left sides of [0.81, 0.79] eye globes, [0.72, 0.79] optic nerves, and [0.73, 0.76] extraocular muscles were achieved. The proposed method is accurate, efficient, does not require training data, and its intuitive pipeline allows the user to modify or extend to other structures.

Objectives: Cricothyrotomy is a life-saving procedure performed when an airway cannot be established through less invasive techniques. The goals of this study were to 1) design and build a low-cost simulator that teaches essential anatomy, and 2) provide a method of data collection for performance evaluation and guided instruction. Methods: The design of the simulator was optimized for materials that are low-cost and widely available and for accurate anatomical landmarks. It was designed to electronically record instrument contact with critical anatomical locations throughout the procedure. Three simulators were constructed, and data were collected during 20 procedures performed by participants of varying surgical experience level. Surveys were completed by all participants to assess the perceived quality of the simulator. Three experts graded the recorded procedures by completing an Objective Structured Assessment of Technical Skills (OSATS) survey. Results: A simulator was designed and built. Procedural data of 20 subjects were collected. The survey data showed the majority of participants answered “moderate” or “very” on questions related to 1) anatomical accuracy (85%), 2) realism (50%), 3) improved understanding of procedure (65%), and 4) useful for practice (85%). The OSATS data correlated with task time. Conclusions: A cricothyrotomy simulator for use in a low-resource environment was designed and produced. It collects data that can be used for performance evaluation and instruction. These were analyzed against OSATS data. A study of 20 subjects, 12 of whom were experts, demonstrated that it was realistic, anatomically accurate, and useful for practice and improving procedure knowledge.

Successful multidisciplinary treatment of skull base pathology requires precise preoperative planning. Current surgical approach (pathway) selection for these complex procedures depends on an individual surgeon's experiences and background training. Because of anatomical variation in both normal tissue and pathology (eg, tumor), a successful surgical pathway used on one patient is not necessarily the best approach on another patient. The question is how to define and obtain optimized patient-specific surgical approach pathways? In this article, we demonstrate that the surgeon's knowledge and decision making in preoperative planning can be modeled by a multiobjective cost function in a retrospective analysis of actual complex skull base cases. Two different approaches- weighted-sum approach and Pareto optimality-were used with a defined cost function to derive optimized surgical pathways based on preoperative computed tomography (CT) scans and manually designated pathology. With the first method, surgeon's preferences were input as a set of weights for each objective before the search. In the second approach, the surgeon's preferences were used to select a surgical pathway from the computed Pareto optimal set. Using preoperative CT and magnetic resonance imaging, the patient-specific surgical pathways derived by these methods were similar (85% agreement) to the actual approaches performed on patients. In one case where the actual surgical approach was different, revision surgery was required and was performed utilizing the computationally derived approach pathway.