Chest L-Transformer: Local Features With Position Attention for Weakly Supervised Chest Radiograph Segmentation and Classification

Abstract

We consider the problem of weakly supervised segmentation on chest radiographs. The chest radiograph is the most common means of screening and diagnosing thoracic diseases. Weakly supervised deep learning models have gained increasing popularity in medical image segmentation. However, these models are not suitable for the critical characteristics presented in chest radiographs: the global symmetry of chest radiographs and dependencies between lesions and their positions. These models extract global features from the whole image to make the image-level decision. The global symmetry can lead these models to misclassification of symmetrical positions of the lesions. Thoracic diseases often have special disease prone areas in chest radiographs. There is a relationship between the lesions and their positions. In this study, we propose a weakly supervised model, called Chest L-Transformer, to take these characteristics into account. Chest L-Transformer classifies an image based on local features to avoid the misclassification caused by the global symmetry. Moreover, associated with Transformer attention mechanism, Chest L-Transformer models the dependencies between the lesions and their positions and pays more attention to the disease prone areas. Chest L-Transformer is only trained with image-level annotations for lesion segmentation. Thus, Log-Sum-Exp voting and its variant are proposed to unify the pixel-level prediction with the image-level prediction. We demonstrate a significant segmentation performance improvement over the current state-of-the-art while achieving competitive classification performance.

Recent Progress in Transformer-based Medical Image Analysis

“…Krishnan and colleagues [28] fine-tuned the transformer to classify X-rays images [53,54]. Gu et al designed a model called Chest L-Transformer [30] to classify chest X-ray images using the SIIM-ACR pneumothorax dataset [55]. The proposed model is composed of a backbone block based on the ResNeXt [56], a position attention block, and a classifier.…”

“…The outputs of the transformers are passed through a weighted fusion layer. [30] 2022 X-ray chest pneumothorax [55] hybrid framework [32] 2021 X-ray chest tuberculosis [57], COVID-19 [58], thorax diseases [59],…”

“…The current frame is fed into the CNN and transformer while the previous frame is fed into the CNN only. The Chest L-Transformer [30] introduced in Table 1 for image classification are also designed for segmentation.…”

“…Due to these, the DL method has been widely applied in the field of medical image analysis to reduce inter-reader variation as well as reduce time and manpower costs. The transformerbased method has been widely used in medical image analysis either using the transformer solely [27,28,29] or hybridizing CNN and transformer to capture both local and global information [30,31,32]. However, an insightful and critical review of transformer-based medical image analysis is absent.…”

Learning a spatial-temporal texture transformer network for video inpainting

“…In recent years, the appearance of the transformer (Arnab et al, 2021 ; Chen et al, 2021 ; Wang and Wang, 2022 ) has provided a new solution for vision tasks. Compared with traditional CNN (Gu et al, 2022 ) and RNN-based (Lin et al, 2022 ) methods, transformers have better capability to understand shape and geometry and capture the dependencies between long distances. We propose a spatial-temporal texture transformer network (Han et al, 2020 ).…”