Unsupervised contrastive learning based transformer for lung nodule detection

Abstract

Objective. Early detection of lung nodules with computed tomography (CT) is critical for the longer survival of lung cancer patients and better quality of life. Computer-aided detection/diagnosis (CAD) is proven valuable as a second or concurrent reader in this context. However, accurate detection of lung nodules remains a challenge for such CAD systems and even radiologists due to not only the variability in size, location, and appearance of lung nodules but also the complexity of lung structures. This leads to a high false-positive rate with CAD, compromising its clinical efficacy. Approach. Motivated by recent computer vision techniques, here we present a self-supervised region-based 3D transformer model to identify lung nodules among a set of candidate regions. Specifically, a 3D vision transformer (ViT) is developed that divides a CT volume into a sequence of non-overlap cubes, extracts embedding features from each cube with an embedding layer, and analyzes all embedding features with a self-attention mechanism for the prediction. To effectively train the transformer model on a relatively small dataset, the region-based contrastive learning method is used to boost the performance by pre-training the 3D transformer with public CT images. Results. Our experiments show that the proposed method can significantly improve the performance of lung nodule screening in comparison with the commonly used 3D convolutional neural networks. Significance. This study demonstrates a promising direction to improve the performance of current CAD systems for lung nodule detection.

Pre-Training, Transfer Learning and Pretext Learning for a Convolutional Neural Network Applied to Automated Assessment of Clinical PET Image Quality

“…Additionally, the model developed here may be used in different applications, including dynamic PET (where shorter frames are desirable for better time resolution, but also lead to noisier data) and different radioligands. Other recent advances, such as self-supervised learning for pre-training, as used by [ 54 ], [ 55 ], or vision-and-language pre-training for the similar task of determining clinical evaluations of medical images [ 56 ] could be investigated in relation to this clinical task.…”

Recent advances of Transformers in medical image analysis: A comprehensive review

“…However, the varied appearance and location of lung nodules make it difficult for automatic computer‐aided detection of lung nodule. To diminish the high false‐positive rate of nodule detection, Niu et al 154 proposed a 3D Transformer framework to achieve lung nodule detection. Specifically, the input CT images got sliced into a nonoverlap sequence, each unit of which was analyzed with selfattention mechanism.…”

“…Specifically, the input CT images got sliced into a nonoverlap sequence, each unit of which was analyzed with selfattention mechanism. Besides, Niu et al 154 chose a region‐based contrastive method to train the model to promote the training result.…”

CT Multi-Task Learning with a Large Image-Text (LIT) Model

“…Swin UNETER [TYL + 22] was preptrained for image segmentation by minimizing the combination of three self-supervised learning losses, involving contrastive learning, inpainting, and rotation. Based on a generic 3D transformer, URCTrans [NW22] was pretrained via contrastive learning, demonstrating its effectiveness for lung nodule detection. However, to our best knowledge, there is no unified large model up to now that works for multiple medical tasks using tomographic scans by integrating various training datasets with diverse annotations.…”