Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset

Groh, Matthew; Harris, Caleb; Soenksen, Luis R.; Lau, F. Din-Houn; Han, Rachel; Kim, Aerin; Koochek, Arash; Badri, Omar

doi:10.1109/cvprw53098.2021.00201

Cited by 109 publications

(35 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dataset and Preprocessing. The proposed methods are evaluated on two dermatology datasets for disease classification, including the Fitzpatrick-17k [7] and ISIC 2019 challenge [2,19] datasets. The Fitzpatrick-17k contains 16,577 images in 114 skin conditions.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…For example, [4,23] demonstrate that the network learned on the CelebA dataset [16] turns to perform better on the female group when the task is predicting facial attributes such as wavy hair and smiling. Dermatological disease classification networks trained on two public dermatology datasets Fitzpatrick-17k and ISIC 2018 Challenge have been reported to be biased across different skin tones [7,12]. However, no solution is proposed to mitigate the bias in these works.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis

Wu¹,

Zeng²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many works have shown that deep learning-based medical image classification models can exhibit bias toward certain demographic attributes like race, gender, and age. Existing bias mitigation methods primarily focus on learning debiased models, which may not necessarily guarantee all sensitive information can be removed and usually comes with considerable accuracy degradation on both privileged and unprivileged groups. To tackle this issue, we propose a method, FairPrune, that achieves fairness by pruning. Conventionally, pruning is used to reduce the model size for efficient inference. However, we show that pruning can also be a powerful tool to achieve fairness. Our observation is that during pruning, each parameter in the model has different importance for different groups' accuracy. By pruning the parameters based on this importance difference, we can reduce the accuracy difference between the privileged group and the unprivileged group to improve fairness without a large accuracy drop. To this end, we use the second derivative of the parameters of a pre-trained model to quantify the importance of each parameter with respect to the model accuracy for each group. Experiments on two skin lesion diagnosis datasets over multiple sensitive attributes demonstrate that our method can greatly improve fairness while keeping the average accuracy of both groups as high as possible.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis

Wu¹,

Zeng²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, erythema is more likely to appear violaceous or just dark brown in darker skin tones, making the delineating of the inflamed border potentially more challenging even if the lesion is not completely missed [16]. Collecting images of eczema on darker skin tones is also relevant for developing ML algorithms that tend to be more accurate on the skin types they were trained on [17]. Secondly, the IRR metrics used in this study did not consider the spatial structure of the image, while neighbouring pixel labels are unlikely to be independent.…”

Section: Discussionmentioning

confidence: 99%

Detecting eczema areas in digital images: an impossible task?

Hurault

Pan

Mokhtari

et al. 2022

Preprint

View full text Add to dashboard Cite

Assessing the severity of atopic dermatitis (AD, or eczema) traditionally relies on a face-to-face assessment by healthcare professionals, and may suffer from inter- and intra-rater variability. With the expanding role of telemedicine, several machine learning algorithms have been proposed to automatically assess AD severity from digital images. Those algorithms usually detect and then delineate (“segment”) AD lesions before assessing lesional severity, and are trained using the data of AD areas detected by healthcare professionals. To evaluate the reliability of such data, we estimated the inter-rater reliability of AD segmentation in digital images.Four dermatologists independently segmented AD lesions in 80 digital images collected in a published clinical trial. We estimated the inter-rater reliability of the AD segmentation using the intra-class correlation coefficients (ICCs) at the pixel-level and the area-levels for different resolutions of the images. The average ICC was 0.45 (SE = 0.04) corresponding to a “poor” agreement between raters, while the degree of agreement for AD segmentation varied from image to image.The AD segmentation in digital images is highly rater-dependent even between dermatologists. Such limitations need to be taken into consideration when the AD segmentation data are used to train machine learning algorithms that assess eczema severity.

show abstract

“…“PAD-UFES-20” is a comparatively new dataset composed in 2020 of almost 2298 images from 1373 using smartphones [ 74 ]. Fitzpatrick Dataset has been developed by Groh et al [ 75 ] that includes almost 17 k clinical images for classifying 114 different skin lesions or cancers. MED-NODE dataset contains 170 images only whereas 70 melanoma cases and 100 nevus cases [ 76 ].…”

Section: Proposed Methodologymentioning

confidence: 99%

A Secure Framework toward IoMT‐Assisted Data Collection, Modeling, and Classification for Intelligent Dermatology Healthcare Services

Islam

Kaushal

Amin

et al. 2022

Contrast Media & Molecular Imaging

View full text Add to dashboard Cite

The abnormal growth of the skin cells is known as skin cancer. It is one of the main problems in the dermatology area. Skin lesions or malignancies have been a source of worry for many individuals in recent years. Irrespective of the skin tone, there exist three major classes of skin lesions, i.e., basal cell carcinoma, squamous cell carcinoma, and melanoma. The early diagnosis of these lesions is equally important for human life. In the proposed work, a secure IoMT-Assisted framework is introduced that can help the patients to do the initial screening of skin lesions remotely. The initially proposed approach uses an IoMT-based data collection device which is accessible by patients to capture skin lesions images. Next, the captured skin sample is encrypted and sent to the collected image toward cloud storage. Later, the received sample image is classified into appropriate class labels using an ensemble classifier. In the proposed framework, four CNN models were ensemble i.e., VGG-16, DenseNet-201, Inception-V3, and Efficient-B7. The framework has experimented with the “HAM10000” dataset having 7 different kinds of skin lesions data. Although DenseNet-201 performed well, the ensemble model provides the highest accuracy with 87.22 percent as well as its test loss/error is lower than others with 0.4131. Moreover, the ensemble model’s classification ability is much higher with an AUC score of 0.9745. Moreover, A recommendation team has been assigned to assess the sample of the patient as well as suggest the patient according to classified results by the CAD.

show abstract

Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset

Cited by 109 publications

References 34 publications

FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis

FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosis

Detecting eczema areas in digital images: an impossible task?

A Secure Framework toward IoMT‐Assisted Data Collection, Modeling, and Classification for Intelligent Dermatology Healthcare Services

Contact Info

Product

Resources

About