AnatomyNet: Deep 3D Squeeze-and-excitation U-Nets for fast and fully automated whole-volume anatomical segmentation

Abstract

Purpose: Radiation therapy (RT) is a common treatment for head and neck (HaN) cancer where therapists are often required to manually delineate boundaries of the organs-at-risks (OARs). The radiation therapy planning is time-consuming as each computed tomography (CT) volumetric data set typically consists of hundreds to thousands of slices and needs to be individually inspected. Automated head and neck anatomical segmentation provides a way to speed up and improve the reproducibility of radiation therapy planning. Previous work on anatomical segmentation is primarily based on atlas registrations, which takes up to hours for one patient and requires sophisticated atlas creation. In this work, we propose the AnatomyNet, an end-to-end and atlas-free three dimensional squeeze-and-excitation U-Net (3D SE U-Net), for fast and fully automated whole-volume HaN anatomical segmentation.Methods: There are two main challenges for fully automated HaN OARs segmentation: 1) challenge in segmenting small anatomies (i.e., optic chiasm and optic nerves) occupying only a few slices, and 2) training model with inconsistent data annotations with missing ground truth for some anatomical structures because of different RT planning. We propose the AnatomyNet that has one down-sampling layer with the trade-off between GPU memory and feature representation capacity, and 3D SE residual blocks for effective feature learning to alleviate these challenges. Moreover, we design a hybrid loss function with the Dice loss and the focal loss. The Dice loss is a class level distribution loss that depends less on the number of voxels in the anatomy, and the focal loss is designed to deal with highly unbalanced segmentation. For missing annotations, we propose masked loss and weighted loss for accurate and balanced weights updating in the learning of the AnatomyNet.Results: We collect 261 HaN CT images to train the AnatomyNet, and use MICCAI Head and Neck Auto Segmentation Challenge 2015 as the benchmark dataset to evaluate the performance of the AnatomyNet. The objective is to segment nine anatomies: brain stem, chiasm, mandible, optic nerve left, optic nerve right, parotid gland left, parotid gland right, submandibular gland left, and submandibular gland right. Compared to previous state-of-the-art methods for each anatomy from the MICCAI 2015 competition, the AnatomyNet increases Dice similarity coefficient (DSC) by 3.3% on average. The proposed AnatomyNet takes only 0.12 seconds on average to segment a whole-volume HaN CT image of an average dimension of 178 × 302 × 225. All the data and code will be available a .Conclusion: We propose an end-to-end, fast and fully automated deep convolutional network, AnatomyNet, for accurate and whole-volume HaN anatomical segmentation. The proposed AnatomyNet outperforms previous state-of-the-art methods on the benchmark dataset. Extensive experiments demonstrate the effectiveness and good generalization ability of the components in the AnatomyNet.

Automatic segmentation of the mandible from computed tomography scans for 3D virtual surgical planning using the convolutional neural network

“…Recent advances in semantic segmentation (Long et al 2015 (Ibragimov & Xing 2017) presented the first attempt of using deep learning concept of CNN to segment organs-at-risks in head and neck CT scans. The AnatomyNet (Zhu, Huang, Tang, Qian, Du, Fan & Xie 2018) is built upon the popular 3D U-net architecture using residual blocks in encoding layers and a new loss function combining Dice score and focal loss in the training process. A fully CNN (FCNN) method was presented by (Tong, Gou, Yang, Ruan & Sheng 2018).…”

FocusNet: Imbalanced Large and Small Organ Segmentation with an End-to-End Deep Neural Network for Head and Neck CT Images

“…Our backbone S-Net achieves state-of-the-art performance. It reaches comparable performance in Dice score to Zhu et al [11] but with only 15% of training data, which shows that S-Net has stronger feature representation capability. Moreover, S-Net has much better result in terms of 95HD, because outliers are alleviated by enlarging the receptive field.…”

“…We compared the highest score from the top four teams in MICCAI 2015 challenge [6]. For the result of Zhu et al [11], it should be noted that they used 38 samples provided by the MICCAI 2015 Challenge combined with additional 216 samples for training.…”

Automated Microfossil Identification and Segmentation Using a Deep Learning Approach

“…With this number of classes, the obtained IOU value went from 40-45 % to 75-76 % and stopped. One problem with this approach is the data balance [31], i.e. , the existence, in the samples, of more annotations from the rock class in comparison with the microfossils.…”