A deep learning system for quantitative assessment of microvascular abnormalities in nailfold capillary images

Abstract

Objectives Nailfold capillaroscopy is key to timely diagnosis of systemic sclerosis (SSc), but is often not used in rheumatology clinics because the images are difficult to interpret. We aimed to develop and validate a fully automated image analysis system to fill this gap. Methods We mimicked the image interpretation strategies of SSc experts, using deep learning networks to detect each capillary in the distal row of vessels and make morphological measurements. We combined measurements from multiple fingers to give a subject-level probability of SSc. We trained the system using high-resolution images from 111 subjects (Group A) and tested on images from subjects not in the training set: 132 imaged at high-resolution (Group B); 66 imaged with a low-cost digital microscope (Group C). Roughly half of each group had confirmed SSc, half were healthy controls or had primary Raynaud’s phenomenon (‘normal’). We also estimated the performance of SSc experts. Results We compared automated SSc probabilities with the known clinical status of patients (SSc versus ‘normal’), generating receiver operating characteristic curves (ROCs). For Group B, the area under the ROC (AUC) was 97% [94% - 99%] (median [90% confidence interval]), with equal sensitivity/specificity 91% [86% - 95%]. For Group C, AUC was 95% [88% - 99%], with equal sensitivity/specificity 89% [82% - 95%]. SSc expert consensus achieved sensitivity 82%, specificity 73%. Conclusion Fully automated analysis using deep learning can achieve diagnostic performance at least as good as SSc experts, and is sufficiently robust to work with low-cost digital microscope images.

Artificial Intelligence Algorithms in Nailfold Capillaroscopy Image Analysis: A Systematic Review

“…Alternatively, Garaiman et al was the only investigator to acknowledge limiting his study to include only diseased patients due to ethical consent considerations (30). The remaining eight studies included both diseased participants and normal controls (15,29,31,32,35,40,41,55). Without including both normal and diseased subjects, an AI algorithm would not be capable of predicting abnormalities.…”

“…Among the six studies investigating systemic sclerosis (SSc), three authors considered two variants as the normal controls: completely disease-free healthy participants and others with a common benign condition known as primary Raynaud's phenomenon (31,35,40); also known as Raynaud's disease (PRD) (56). Conversely, Garaiman et al and Tello et all were satisfied with PRD only as normal controls (15,29,30).…”

“…Authors differed in their approach to obtaining sample images from participants, and in how balanced the normal controls and main study groups were. For instance, five authors obtained multiple images from multiple sections of multiple nailfolds spanning multiple encounters (28,30,31,41,55)…”

“…Four authors adopted the second approach of 'classification' that aimed to classify the input image into one category out of two or more categories/classes; either at the pixel, capillary, image, or global levels (30,40,41,55). The remaining three authors took advantage of a multi-step mixed approach that relied on detecting certain features first, then classifying them at one or more levels into the assigned classes (15,29,31,35).…”

“…To illustrate, many deep learning models have demonstrated the ability to objectively analyze images from NFC/NVC with higher, or at least comparable, efficiency as capillaroscopy experts in a significantly lower time (5,15,(28)(29)(30)(31)(32)(33)(34)(35)(36)(37)(38)(39)(40)(41). Advances in computer vision algorithms allow the extraction, quantification, and accurate analysis of far more features compared to human experts.…”