Remote Assessment of Acne: The Use of Acne Grading Tools to Evaluate Digital Skin Images

Abstract

Digital imaging of dermatology patients is a novel approach to remote data collection. A number of assessment tools have been developed to grade acne severity and to track clinical progress over time. Although these tools have been validated when used in a face-to-face setting, their efficacy and reliability when used to assess digital images have not been examined. The main purpose of this study was to determine whether specific assessment tools designed to grade acne during face-to-face visits can be applied to the evaluation of digital images. The secondary purpose was to ascertain whether images obtained by subjects are of adequate quality to allow such assessments to be made. Three hundred (300) digital images of patients with mild to moderate facial inflammatory acne from an ongoing randomized-controlled study were included in this analysis. These images were obtained from 20 patients and consisted of sets of 3 images taken over time. Of these images, 120 images were captured by subjects themselves and 180 were taken by study staff. Subjects were asked to retake their photographs if the initial images were deemed of poor quality by study staff. Images were evaluated by two dermatologists-in-training using validated acne assessment measures: Total Inflammatory Lesion Count, Leeds technique, and the Investigator's Global Assessment. Reliability of raters was evaluated using correlation coefficients and kappa statistics. Of the different acne assessment measures tested, the inter-rater reliability was highest for the total inflammatory lesion count (r = 0.871), but low for the Leeds technique (kappa = 0.381) and global assessment (kappa = 0.3119). Raters were able to evaluate over 89% of all images using each type of acne assessment measure despite the fact that images obtained by study staff were of higher quality than those obtained by patients (p < 0.001). Several existing clinical assessment measures can be used to evaluate digital images obtained from subjects with inflammatory acne lesions. The level of inter-rater agreement is highly variable across assessment measures, and we found the Total Inflammatory Lesion Count to be the most reliable. This measure could be used to allow a dermatologist to remotely track a patient's progress over time.

International inter‐rater agreement in scoring acne severity utilizing cloud‐based image sharing of mobile phone photographs

“…7 Interestingly, we found there to be good agreement in counting NILs as well, especially among graders from the same institution (KCC = 0.76, When counting individual lesions, good agreement was found for the evaluation of ILs, in agreement with a previous photographic study that utilized separate digital cameras instead of mobile phone-based photography.…”

“…When counting individual lesions, good agreement was found for the evaluation of ILs, in agreement with a previous photographic study that utilized separate digital cameras instead of mobile phone-based photography. 7 Interestingly, we found there to be good agreement in counting NILs as well, especially among graders from the same institution (KCC = 0.76, P < 0.0001).…”

“…7 More specifically, one study found a Pearson's correlation coefficient of 0.871, showing inter-rater agreement for total inflammatory lesion count and a kappa score of 0.3119 for investigator's global assessment score showing low inter-rater agreement for the ratings given by two dermatologists. 7 More specifically, one study found a Pearson's correlation coefficient of 0.871, showing inter-rater agreement for total inflammatory lesion count and a kappa score of 0.3119 for investigator's global assessment score showing low inter-rater agreement for the ratings given by two dermatologists.…”

“…Previous studies of global evaluation have been mixed with good agreement in some global grading scales 8 while poor in other ones. 7 More specifically, one study found a Pearson's correlation coefficient of 0.871, showing inter-rater agreement for total inflammatory lesion count and a kappa score of 0.3119 for investigator's global assessment score showing low inter-rater agreement for the ratings given by two dermatologists. 7 In another study investigating inter-rater agreement for in-person assessment of acne, the authors found inter-class coefficient correlations of 0.84-0.98, 0.61-0.95, and 0.43-0.91 for noninflammatory lesion counts, inflammatory lesion counts, and global assessment score.…”

“…7 More specifically, one study found a Pearson's correlation coefficient of 0.871, showing inter-rater agreement for total inflammatory lesion count and a kappa score of 0.3119 for investigator's global assessment score showing low inter-rater agreement for the ratings given by two dermatologists. 7 In another study investigating inter-rater agreement for in-person assessment of acne, the authors found inter-class coefficient correlations of 0.84-0.98, 0.61-0.95, and 0.43-0.91 for noninflammatory lesion counts, inflammatory lesion counts, and global assessment score. 9 Our inter-rater agreement was similar to previous studies in regards to the inflammatory lesion counts, Figure 4 Correlation between the subjects' median global acne grading system scores and the number of dermatologists recommending oral antibiotics or oral isotretinoin.…”