Fully transformer network for skin lesion analysis

He, Xinzi; Tan, Eng Leong; Bi, Hanwen; Zhang, Xuzhe; Zhao, Shijie; Lei, Baiying

doi:10.1016/j.media.2022.102357

Cited by 75 publications

(26 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent studies have proposed numerous advances to improve the classification accuracy of skin lesions. On the architectural front, advanced methods include wavelet domain CNN models [32], [33], synergic models that contain an ensemble of CNNs [34], multi-tasking models that leverage dermoscopy images along with their segmentation features [16], attention-gated CNN or self-attention transformer models [9], [35], [36]. On the algorithmic front, proposed techniques include domain transfer of pre-trained feature sets [37], augmentation via GAN-based synthetic sample generation [38], [39], and combination of multiple imaging modalities and patient metadata [40].…”

Section: Related Workmentioning

confidence: 99%

Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets

Öztürk

Çukur

2022

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Melanoma is a fatal skin cancer that is curable and has dramatically increasing survival rate when diagnosed at early stages. Learning-based methods hold significant promise for the detection of melanoma from dermoscopic images. However, since melanoma is a rare disease, existing databases of skin lesions predominantly contain highly imbalanced numbers of benign versus malignant samples. In turn, this imbalance introduces substantial bias in classification models due to the statistical dominance of the majority class. To address this issue, we introduce a deep clustering approach based on the latentspace embedding of dermoscopic images. Clustering is achieved using a novel center-oriented margin-free triplet loss (COM-Triplet) enforced on image embeddings from a convolutional neural network backbone. The proposed method aims to form maximally-separated cluster centers as opposed to minimizing classification error, so it is less sensitive to class imbalance. To avoid the need for labeled data, we further propose to implement COM-Triplet based on pseudo-labels generated by a Gaussian mixture model (GMM). Comprehensive experiments show that deep clustering with COM-Triplet loss outperforms clustering with triplet loss, and competing classifiers in both supervised and unsupervised settings.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets

Öztürk

Çukur

2022

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

show abstract

“…The attention mechanism is a biomimetic cognitive method used in diverse computer vision assignments such as image classification (Woo et al, 2018;Hu et al, 2018;Dosovitskiy et al, 2020;Hou et al, 2021;Mehta and Rastegari, 2021) and image segmentation (Mehta and Rastegari, 2021;He et al, 2022). An example of the attention network is the SENet, which obtains global representations by global average pooling and channel-wise feature response by squeeze and excitation (Hu et al, 2018).…”

Section: Attention Mechanismsmentioning

confidence: 99%

“…However, the standard transformer method ignores the inductive biases (e.g., translation equivariance and locality) inherent to CNNs, which leads to poor performance while training with insufficient data (Dosovitskiy et al, 2020). Recently, (He et al, 2022) first introduced a purely ViT based network to analyse skin lesions with acceptable results on segmentation and classification. Although their pyramid pooling in the multihead self-attention (He et al, 2022) has linear computational complexity, the model is still significantly sizeable with 8M to 19M parameters.…”

Section: Vision Transformermentioning

confidence: 99%

“…Recently, (He et al, 2022) first introduced a purely ViT based network to analyse skin lesions with acceptable results on segmentation and classification. Although their pyramid pooling in the multihead self-attention (He et al, 2022) has linear computational complexity, the model is still significantly sizeable with 8M to 19M parameters. (Wu et al, 2022) also utilised ViT in ski lesions evaluation by appending a ViT branch to CNN based Ushape architecture for improving long-range dependencies and contextual information extraction, but an additional branch notably increased the model weight.…”

Section: Vision Transformermentioning

confidence: 99%

“…Modified ResNet with U-shape CNN architectures were utilised to complete classification and segmentation tasks simultaneously (Chen et al, 2018;Xie et al, 2020). (He et al, 2022) further advanced this idea by replacing CNNs with vision transformers (ViTs) to capture contextual information and long-range dependency. Additionally, random patches dropout was taken to reduce the influence of relative position and contribute to more discriminative high-level features (Gessert et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

HierAttn: Effectively Learn Representations from Stage Attention and Branch Attention for Skin Lesions Diagnosis

Dai¹,

Li²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate and unbiased examinations of skin lesions are critical for the early diagnosis and treatment of skin conditions and disorders. Visual features of skin lesions vary significantly because the images are collected from patients with different lesion colours and morphologies by using dissimilar imaging equipment. Recent studies have reported ensembled convolutional neural networks (CNNs) to classify the images for early diagnosis of skin disorders. However, the practical use of these ensembled CNNs is limited because they are heavyweight and inadequate for using contextual information. Although lightweight networks (e.g., MobileNetV3 and EfficientNet) were developed to achieve parameters reduction for implementing deep neural networks on mobile devices, insufficient depth of feature representation restricts the performance. To address the existing limitations, we introduce a new lite and effective neural network, namely HierAttn. The HierAttn applies a novel strategy to learn the local and global features by using multi-stage and multi-branch attention mechanisms. The efficacy of Hier-Attn was evaluated by using the dermoscopy images dataset ISIC2019 and smartphone photos dataset PAD-UFES-20 (PAD20). The experimental results show that HierAttn achieves the best accuracy and AUC among the state-of-the-art lightweight networks. The code is available at https://github.com/anthonyweidai/HierAttn.

show abstract

A Dynamic Multi‐Output Convolutional Neural Network for Skin Lesion Classification

Zhou,

Guo,

Yao

et al. 2024

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Skin cancer is a pressing global health issue, with high incidence and mortality rates. Convolutional neural network (CNN) models have been proven to be effective in improving performance in skin lesion image classification and reducing the medical burden. However, the inherent class imbalance in training data caused by the difficulty of collecting dermoscopy images leads to categorical overfitting, which still limits the effectiveness of these data‐driven models in recognizing few‐shot categories. To address this challenge, we propose a dynamic multi‐output convolutional neural network (DMO‐CNN) model that incorporates exit nodes into the standard CNN structure and includes feature refinement layers (FRLs) and an adaptive output scheduling (AOS) module. This model improves feature representation ability through multi‐scale sub‐feature maps and reduces the inter‐layer dependencies during backpropagation. The FRLs ensure efficient and low‐loss down‐sampling, while the AOS module utilizes a trainable layer selection mechanism to refocus the model's attention on few‐shot lesion categories. Additionally, a novel correction factor loss is introduced to supervise and promote AOS convergence. Our evaluation of the DMO‐CNN model on the HAM10000 dataset demonstrates its effectiveness in multi‐class skin lesion classification and its superior performance in recognizing few‐shot categories. Despite utilizing a very simple VGG structure as the sole backbone structure, DMO‐CNN achieved impressive performance of 0.885 in BACC and 0.983 in weighted AUC. These results are comparable to those of the ensemble model that won the ISIC 2018 challenge, highlighting the strong potential of DMO‐CNN in dealing with few‐shot skin lesion data.

show abstract

Fully transformer network for skin lesion analysis

Cited by 75 publications

References 28 publications

Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets

Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets

HierAttn: Effectively Learn Representations from Stage Attention and Branch Attention for Skin Lesions Diagnosis

A Dynamic Multi‐Output Convolutional Neural Network for Skin Lesion Classification

Contact Info

Product

Resources

About