Automatic Vertebra Labeling in Large-Scale 3D CT Using Deep Image-to-Image Network with Message Passing and Sparsity Regularization

Abstract

Abstract. Automatic localization and labeling of vertebra in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. However, the unusual conditions of pathological cases, such as the abnormal spine curvature, bright visual imaging artifacts caused by metal implants, and the limited field of view, increase the difficulties of accurate localization. In this paper, we propose an automatic and fast algorithm to localize and label the vertebra centroids in 3D CT volumes. First, we deploy a deep image-to-image network (DI2IN) to initialize vertebra locations, employing the convolutional encoder-decoder architecture together with multi-level feature concatenation and deep supervision. Next, the centroid probability maps from DI2IN are iteratively evolved with the message passing schemes based on the mutual relation of vertebra centroids. Finally, the localization results are refined with sparsity regularization. The proposed method is evaluated on a public dataset of 302 spine CT volumes with various pathologies. Our method outperforms other state-of-the-art methods in terms of localization accuracy. The run time is around 3 seconds on average per case. To further boost the performance, we retrain the DI2IN on additional 1000 + 3D CT volumes from different patients. To the best of our knowledge, this is the first time more than 1000 3D CT volumes with expert annotation are adopted in experiments for the anatomic landmark detection tasks. Our experimental results show that training with such a large dataset significantly improves the performance and the overall identification rate, for the first time by our knowledge, reaches 90 %.

Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information

“…It is considered challenging due to the variety of pathological cases, arbitrary of field-of-view and the existence of artificial implants. For all the experiments, we use the official split for training and testing as did by other stateof-the-art methods [19], [20], [24]. In total, there are 302 CT scans in this dataset.…”

“…However, as denoted in [23] and also demonstrated in this work, 2D CNNs do not work well in detection problems as they cannot capture the 3D spatial information that is critical to the detection of the target object. More recently, Dong et al [24] proposed a 3D U-Net [25] like architecture to target the vertebrae localization problem in an image-to-image fashion. However, the proposed architecture can not fully address the long-term contextual information in spinal images.…”

Btrfly Net: Vertebrae Labelling with Energy-Based Adversarial Learning of Local Spine Prior

“…Performance comparison of our approach (setting T = 0, for a fair comparison) with Glocker et al[3], Chen et al[1], & Yang et al[8]. DI2IN refers to stand-alone FCN, while DI2IN* includes use of message passing and shape dictionary.…”

How Generative Adversarial Networks and Their Variants Work

“…Music generation C-RNN-GAN [83], SeqGAN [141], ORGAN [41] Text generation RankGAN [73] Speech conversion VAW-GAN [48] Semi-supervised learning SSL-GAN [104], CatGAN [115], Triple-GAN [67] Others Domain adaptation DANN [2], CyCADA [47] Unsupervised pixel-level domain adaptation [12] Continual learning Deep generative replay [110] Medical image segmentation DI2IN [136], SCAN [16], SegAN [134] Steganography Steganography GAN [124], Secure steganography GAN [109] is more likely to be real. G and D compete with each other to achieve their individual goals, thus generating the term adversarial.…”

“…This structure leads a segmentor to learn the features of the ground-truth segmentation adversarially similar to the GAN approach. There are also other medical image segmentation algorithms such as the deep image-to-image network (DI2IN) [136] and structure correcting adversarial network (SCAN) [16]. DI2IN conducts liver segmentation of 3D CT images through adversarial learning.…”