3D Segmentation of Necrotic Lung Lesions in CT Images Using Self-Supervised Contrastive Learning

Abstract

Deep convolutional neural networks (CNN) are often trained on 2D annotations created by radiologists following RECIST guidelines to segment lesions in 3D medical images. Three-dimensional segmentation is conducted by segmenting each lesion slice-by-slice on the axial direction and stacking the 2D segmentation masks into 3D. However, the performance of such models is inherently biased by the appearance of most of the lesions in the training dataset. Herein we propose an approach to generate accurate 3D segmentations of underrepresented necrotic lung lesions. Our proposed approach applies two novel augmentation techniques for contrastive learning pretraining: dependency augmentation that captures inter-slice and inter-timepoint dependencies, and distance transform-based mask-out augmentation imitating necrotic lesions. In dependency augmentation, cosine similarity within RECIST bounding box is applied to construct positive pairs from 2D image slices of the same lesion in the current 3D volume and across longitudinal scans. We further compared contrastive learning architectures, Momentum Contrast (MoCo) and Bootstrap Your Own Latent (BYOL), based upon two internal 3D testing sets, one with regular lung lesions and the other with necrotic lung lesions, and a public 3D DeepLesion lung lesion testing set. MoCo with both proposed augmentations demonstrated the best performance among all methods that were compared. Specifically, it 1) improved Dice similarity coefficient (DSC) by 8.42% over baseline model trained from scratch and 2.40% over ImageNet pretrained model on the 3D necrotic lung lesion set; 2) achieved better segmentation performance on necrotic lesions with 10% of labeled data for supervised fine-tuning compared with the baseline model trained with all labels from scratch.