Few‐shot learning for dermatological conditions with Lesion Area Aware Swin Transformer

Abstract

Skin is the largest organ of the human body and participates in the functional activities of the human body all the time. Therefore, human beings have a large risk of getting skin diseases. The diseased skin lesion image shows visually different characteristics from the normal skin image, and sometimes unusual skin color may indicate human viscera or autoimmune issues. However, the current recognition and classification of dermatological conditions still rely on expert visual diagnosis rather than a visual algorithm. This is because there are many kinds of lesion features of skin diseases, and the lesion accounts for a small proportion of the skin image, so it is difficult to learn the required lesion features; meanwhile, some dermatology images have too few samples to deal with the problem of small samples. In view of the above limitations, we propose a model named Lesion Area Aware Shifted windows Transformer for dermatological conditions classification rely on the powerful performance and excellent result of Swin transformer recently proposed. For brief notation, we use its abbreviation later. Our main contributions are as follows. First, we modify the Swin transformer and use it in the automatic classification of dermatological conditions. Using the self‐attention mechanism of the transformer, our method can mine more long‐distance correlations between diseased tissue image features. Using its shifting windows, we can fuse local features and global features, so it is possible to get better classification results with a flexible receptive field. Second, we use a skip connection to grasp and reinforce global features from the previous block and use Swin transformer to extract detailed local features, which will excavate and merge global features and local features further. Third, considering Swin transformer is a lightweight model compared with traditional transformers, our model is compact for deployment and more favorable to resource‐strict medical devices.

SkinLiTE: Lightweight Supervised Contrastive Learning Model for Enhanced Skin Lesion Detection and Disease Typification in Dermoscopic Images

“…The attention branch worked in conjunction with a global branch (which learned patterns from the entire image) and a fusion branch (which integrated knowledge from both global and local branches). Ren, et al [113] modified the Swin transformer for the automatic classification of dermatological conditions, using a self-attention mechanism and shifting windows. Finally, Zhang, et al [114] developed CR-Conformer, a dual-branch fusion network integrating transformer branches for classifying clinical skin lesion images.…”