Computer-aided diagnosis of retinopathy based on vision transformer

Abstract

Age-related Macular Degeneration (AMD) and Diabetic Macular Edema (DME) are two common retinal diseases for elder people that may ultimately cause irreversible blindness. Timely and accurate diagnosis is essential for the treatment of these diseases. In recent years, computer-aided diagnosis (CAD) has been deeply investigated and effectively used for rapid and early diagnosis. In this paper, we proposed a method of CAD using vision transformer to analyze optical coherence tomography (OCT) images and to automatically discriminate AMD, DME, and normal eyes. A classification accuracy of 99.69% was achieved. After the model pruning, the recognition time reached 0.010 s and the classification accuracy did not drop. Compared with the Convolutional Neural Network (CNN) image classification models (VGG16, Resnet50, Densenet121, and EfficientNet), vision transformer after pruning exhibited better recognition ability. Results show that vision transformer is an improved alternative to diagnose retinal diseases more accurately.

A vision transformer architecture for the automated segmentation of retinal lesions in spectral domain optical coherence tomography images

“…C.1 —In adapting Swin-UNETR to the domain of automatic retinal OCT lesion segmentation, this work contributed to a recent research strand that focuses on the use of ViTs for the automated segmentation of retinal lesions in OCT images. In particular, in our study, we used a hybrid Transformer-CNN that, unlike the pure transformer used in recent contributions 25 , 26 , is less training data demanding and thus does not require pretraining. In this work, the adaptation of the Swin-UNETR is limited to experimenting with different feature sizes.…”

“…(2) A CNN-based feature extractor produces a semantic gap that is subsequently bridged by adaptively fusing multi-scale features, while our study assumes a weaker semantic gap thanks to a transformer-based feature extractor. Other relevant studies are the ones of Kihara 25 and Jiang 26 , where transformers have been considered for analyzing OCT images. Kihara and coauthors focused on nonexudative macular neovascularization and proposed a transformer-based encoder-decoder architecture for the segmentation task.…”

“…In our study, we rely on a hybrid Transformer-CNN that, unlike the pure transformer used by Kihara, is less training data demanding and thus does not require pretraining. In particular, we evaluate the use of a shifted window transformer backbone which, contrary to the transformer backbones in the aforementioned studies 25 , 26 , can efficiently process more fine-grained image patches through the use of local self-attention. This follows the concept exploited in the work of Playout 23 , which increased the number of patches drawn from the image through convolutions with adjustable strides to improve classification performance and used focused attention to compensate for the additional memory requirements.…”

Landslide Susceptibility Mapping by Fusing Convolutional Neural Networks and Vision Transformer

“…After feature extraction, the Classtoken is fully concatenated for classification using MHSA ( Figure 4 ). In this paper, the ViT-B/16 model, a derivative of ViT, is chosen, consisting of a stack of 12 blocks, each containing 16 attentional mechanisms [ 38 , 39 ].…”

Application of Transformer Models to Landslide Susceptibility Mapping

“…CNN models use convolutional kernels to extract features from a fixed range of data, which can also be referred to as local feature extraction. Vision Transformer (ViT) models, using the encoder from a Transformer Encoder base model, have achieved good results in the field of image classification [ 47 , 48 , 49 ]. Feature extraction with ViT differs from CNN in that there is more similarity between the representations obtained at shallow and deep layers.…”