Skin Disease Recognition Using Deep Saliency Features and Multimodal Learning of Dermoscopy and Clinical Images

Ge, Zongyuan; Demyanov, Sergey; Chakravorty, Rajib; Bowling, Adrian; Garnavi, Rahil

doi:10.1007/978-3-319-66179-7_29

Cited by 93 publications

(54 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, Ge et al described the automated analysis of clinical and dermatoscopic images by using the average output of two separate networks, siamese networks, or training of a third network of fused feature maps. They achieved up to 8% increase in accuracy at a multi‐task problem, with their data set incorporating expert‐labelled data without pathologically verified cases.…”

Section: Discussionmentioning

confidence: 99%

Multimodal skin lesion classification using deep learning

Yap

Yolland

Tschandl

2018

Experimental Dermatology

223

122

View full text Add to dashboard Cite

While convolutional neural networks (CNNs) have successfully been applied for skin lesion classification, previous studies have generally considered only a single clinical/macroscopic image and output a binary decision. In this work, we have presented a method which combines multiple imaging modalities together with patient metadata to improve the performance of automated skin lesion diagnosis. We evaluated our method on a binary classification task for comparison with previous studies as well as a five class classification task representative of a real-world clinical scenario. We showed that our multimodal classifier outperforms a baseline classifier that only uses a single macroscopic image in both binary melanoma detection (AUC 0.866 vs 0.784) and in multiclass classification (mAP 0.729 vs 0.598). In addition, we have quantitatively showed the automated diagnosis of skin lesions using dermatoscopic images obtains a higher performance when compared to using macroscopic images. We performed experiments on a new data set of 2917 cases where each case contains a dermatoscopic image, macroscopic image and patient metadata.

show abstract

Section: Discussionmentioning

confidence: 99%

Multimodal skin lesion classification using deep learning

Yap

Yolland

Tschandl

2018

Experimental Dermatology

223

122

View full text Add to dashboard Cite

show abstract

“…A large amount of works on medical images builds on the CAM technique. Examples include the work of Feng et al [12] on pulmonary nodule localisation in CT, the work of Ge et al [18] on skin disease recognition. Other examples are [69], [19].…”

Section: Related Work 21 Visual Attributionmentioning

confidence: 99%

Visual Feature Attribution Using Wasserstein GANs

Baumgartner

Koch

Tezcan

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

128

188

View full text Add to dashboard Cite

Attributing the pixels of an input image to a certain category is an important and well-studied problem in computer vision, with applications ranging from weakly supervised localisation to understanding hidden effects in the data. In recent years, approaches based on interpreting a previously trained neural network classifier have become the de facto state-of-the-art and are commonly used on medical as well as natural image datasets. In this paper, we discuss a limitation of these approaches which may lead to only a subset of the category specific features being detected. To address this problem we develop a novel feature attribution technique based on Wasserstein Generative Adversarial Networks (WGAN), which does not suffer from this limitation. We show that our proposed method performs substantially better than the state-of-the-art for visual attribution on a synthetic dataset and on real 3D neuroimaging data from patients with mild cognitive impairment (MCI) and Alzheimer's disease (AD). For AD patients the method produces compellingly realistic disease effect maps which are very close to the observed effects.

show abstract

“…The modification makes the solution update to be more efficient, and it further improves the convergence of the optimisation algorithm. The velocity of the bat at s + 1 th iteration is given as (9) where E * refer to the best position of the bat, p u signifies the frequency of the uth bat, and h u s indicates the velocity of the bat in sth iteration. Then, (8) is modified using the SSD to improve the effectiveness of the approach and to identify the solutions to several optimisation problems.…”

Section: (I) Fitness Function Computationmentioning

confidence: 99%

Social bat optimisation dependent deep stacked auto‐encoder for skin cancer detection

Majji¹,

Prakash

Cristin³

et al. 2020

IET image process

View full text Add to dashboard Cite

Nowadays, skin cancer is one of the most dangerous forms of cancer found in humans. There are various types of skin cancer, like basal, melanoma, carcinoma, and the squamous cell from which the melanoma is unpredictable. Thus, skin cancer detection in the early stage is very useful to treat it successfully. Hence, this study introduces a new algorithm called social bat optimisation algorithm for skin cancer detection. Initially, the pre-processing is done for the input image to eliminate the noise and artefacts present in the image. Then, the pre-processed image is fed to the feature extraction step where the features are extracted based on convolutional neural network features, and the local pixel pattern-based texture feature (local PPBTF). Here, the PPBTF is the combination of texture features and pixel pattern-based features in which the equation of PPBTF is modified based on the local binary pattern. Subsequently, the classification is done based on the extracted features using a deep stacked auto-encoder, which is trained by the proposed social bat optimisation. The performance of skin cancer detection based on the proposed model is evaluated based on accuracy, sensitivity, and specificity. The proposed model achieves the maximal accuracy of 93.38%, maximal sensitivity of 95%, and the maximal specificity of 96% for K-fold.

show abstract

Skin Disease Recognition Using Deep Saliency Features and Multimodal Learning of Dermoscopy and Clinical Images

Cited by 93 publications

References 11 publications

Multimodal skin lesion classification using deep learning

Multimodal skin lesion classification using deep learning

Visual Feature Attribution Using Wasserstein GANs

Social bat optimisation dependent deep stacked auto‐encoder for skin cancer detection

Contact Info

Product

Resources

About