Vision transformer assisting rheumatologists in screening for capillaroscopy changes in systemic sclerosis: an artificial intelligence model

Garaiman, Alexandru; Nooralahzadeh, Farhad; Mihai, Carina; Gonzalez, Nicolas Perez; Gkikopoulos, Nikitas; Becker, Mike; Distler, Oliver; Krauthammer, Michael; Maurer, Britta

doi:10.1093/rheumatology/keac541

Cited by 16 publications

(29 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AUTOCAPI measures capillary density for a single field of view (preselected by the operator) [ 16 ]. Capillary.io [ 13 ] and a system developed by Swiss researchers [ 14 ] both detect vessels and signs of abnormality (e.g. giant capillaries, microhaemorrhages) from a single field of view (as opposed to a nailfold mosaic).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A deep learning system for quantitative assessment of microvascular abnormalities in nailfold capillary images

Bharathi

Berks

Dinsdale³

et al. 2023

Rheumatology

View full text Add to dashboard Cite

Objectives Nailfold capillaroscopy is key to timely diagnosis of systemic sclerosis (SSc), but is often not used in rheumatology clinics because the images are difficult to interpret. We aimed to develop and validate a fully automated image analysis system to fill this gap. Methods We mimicked the image interpretation strategies of SSc experts, using deep learning networks to detect each capillary in the distal row of vessels and make morphological measurements. We combined measurements from multiple fingers to give a subject-level probability of SSc. We trained the system using high-resolution images from 111 subjects (Group A) and tested on images from subjects not in the training set: 132 imaged at high-resolution (Group B); 66 imaged with a low-cost digital microscope (Group C). Roughly half of each group had confirmed SSc, half were healthy controls or had primary Raynaud’s phenomenon (‘normal’). We also estimated the performance of SSc experts. Results We compared automated SSc probabilities with the known clinical status of patients (SSc versus ‘normal’), generating receiver operating characteristic curves (ROCs). For Group B, the area under the ROC (AUC) was 97% [94% - 99%] (median [90% confidence interval]), with equal sensitivity/specificity 91% [86% - 95%]. For Group C, AUC was 95% [88% - 99%], with equal sensitivity/specificity 89% [82% - 95%]. SSc expert consensus achieved sensitivity 82%, specificity 73%. Conclusion Fully automated analysis using deep learning can achieve diagnostic performance at least as good as SSc experts, and is sufficiently robust to work with low-cost digital microscope images.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The number of capillaries divided by the distance in mm from the left-most to right-most capillary was computed to quantify vessel density. Other investigators have included micro-haemorrhages [ 13 , 14 ]; we did not because they are difficult to define and are not necessary for diagnosis of SSc [ 1 ].…”

Section: Methodsmentioning

confidence: 99%

A deep learning system for quantitative assessment of microvascular abnormalities in nailfold capillary images

Bharathi

Berks

Dinsdale³

et al. 2023

Rheumatology

View full text Add to dashboard Cite

show abstract

“…22 The proximal nailfold is a preferred location for evaluation of the capillary loops because in this location they run parallel to the skin surface, enabling better visualization. 23 Also the scalp may fulfill this criterion of easy visualization of capillaries. Kwiatkowska et al 24 reported that vessels found on trichoscopy of 17 patients with systemic sclerosis were more commonly arranged into random, linear and clustered pattern in comparison to healthy controls.…”

Section: Syste M Ic Scl E Rosismentioning

confidence: 99%

“…Nailfold capillaroscopy is a well established method of assessing microangiopathy in the course of systemic sclerosis 22 . The proximal nailfold is a preferred location for evaluation of the capillary loops because in this location they run parallel to the skin surface, enabling better visualization 23 . Also the scalp may fulfill this criterion of easy visualization of capillaries.…”

Section: Systemic Sclerosismentioning

confidence: 99%

The role of trichoscopy beyond hair and scalp diseases: A review

Rudnicka

Chrostowska

Kamiński

et al. 2023

Acad Dermatol Venereol

View full text Add to dashboard Cite

Trichoscopy is a diagnostic tool for hair and scalp diseases. It was recently shown that it also allows the identification of features associated with disorders that typically do not affect the scalp. The aim of this article was to analyse and outline the usefulness of trichoscopy in suspecting such diseases. Connective tissue diseases were the most investigated systemic disorders in regard to trichoscopy. The most common features of systemic lupus erythematosus, systemic sclerosis and dermatomyositis are thick arborizing and tortuous vessels. Avascular areas are present in systemic sclerosis. Spermatozoa‐like vessels may be observed in cutaneous T‐cell lymphomas, while salmon‐coloured areas with arborizing and linear vessels may be seen in patients with cutaneous B‐cell lymphomas. In patients with advanced multiple myeloma, follicular spicules may be observed. Trichoscopic features of angiosarcomas include pink areas, red, polymorphic areas and dark red to purple areas. Polymorphous vessels and whitish areas on a pink background are the predominating trichoscopic features of metastases of malignant tumours to the scalp. Cutaneous sarcoidosis is characterized by orange‐coloured areas and telangiectasias. Systemic amyloidosis may manifest with salmon‐coloured perifollicular halos, while the most common trichoscopic features of syphilitic alopecia are as follows: decreased number of hairs per follicular unit, vellus hairs, background erythema, focal atrichia and yellow dots. In conclusion, dermatologists may suspect some systemic diseases on the basis of trichoscopic findings.

show abstract

“…Furthermore, ML frameworks were developed to predict rapid coronary plaque progression to identify patients at risk, as well as to assess the importance of clinical parameters [17]. Similarly, Garaiman et al assessed the performance of vision transformer (deep learning image based model) for identifying distinct signs of microangiopathy using nailfold capilloroscopy images of patients with SSc [18].…”

Section: Introductionmentioning

confidence: 99%

Predicting interstitial lung disease progression in patients with systemic sclerosis using attentive neural processes - a EUSTAR study

Allam,

Horvath,

Dittberner

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Systemic sclerosis (SSc) is an autoimmune disease with high mortality with lung involvement being the primary cause of death. Progressive interstitial lung disease (ILD) leads to a decline in lung function (forced vital capacity, FVC% predicted) with risk of respiratory failure. These patients could benefit from an early and tailored pharmacological intervention. However, up to date, tools for prediction of individual FVC changes are lacking. In this paper, we aimed at developing a trustworthy machine learning system that is able to guide SSc management by providing not only robust FVC predictions, but also uncertainty quantification (i.e. the degree of certainty of model prediction) as well as similarity-based explainability for any patient P (i.e. a list of past SSc patients with similar FVC trajectories like P). We further aimed to identify the key clinical factors influencing the model's predictions and to use model-guided data representation to identify SSc patients with similar sequential FVC measurements. Methods: We trained and evaluated machine learning (ML) models to predict SSc-ILD trajectory as measured by FVC% predicted values using the international SSc database managed by the European Scleroderma Trials and Research group (EUSTAR), which comprises clinical, laboratory and functional parameters. EUSTAR records patients' data in annual assessment visits, and, given any visit, we aimed at predicting the FVC value of a patient's subsequent visit, taking into account all available patient data (i.e. baseline and follow-up visit data up to the time point where we make the prediction). For the training of our ML models, we included 2220 SSc patients that had at least 3 recorded visits in the EUSTAR database, were at least 18 years old, had confirmed ILD and sufficient clinical documentation. We developed sequential ML models implementing the attentive neural process formalism with either a recurrent (ANP RNN) or transformer encoder (ANP transformer) architecture. We compared these architectures with baseline sequential models including gated recurrent neural networks (RNNs) and multi-head self-attention transformer-based networks. Baseline non-sequential models included tree-based models such as gradient boosting trees, and regression-based models with varying regularization schemes. Our experiments used stratified 5-fold cross-validation to train and test the models using the average root mean squared error (RMSE), weighted RMSE, and mean absolute error (MAE) as performance metrics. We computed the coverage and Winkler score for uncertainty quantification, SHAP values for grading the input features importance and used the data embeddings of the ANP architectures for both similarity-based explainability and the identification of similar SSc patient journeys. Results: Patients' baseline FVC scores ranged from 22 to 150% predicted with a mean (SD) of 90.53% predicted (21.52). Our deep learning models showed better performance for FVC forecasting, compared to tree- and regression-based models. The top performing ANP RNN architecture was able to closely model future FVC values with average (SD) performance of 8.240 (0.168) weighted RMSE and 6.94 (0.190) MAE that was further used as feature generator for a logistic regression trained to predict a FVC% decline of at least 10% points achieving 0.704 AUC score. In comparison, a naive baseline using the mean FVC value as a predictor achieved much lower FVC forecasting capabilities, with 18.718 (0.317) weighted RMSE, and 17.619 (0.599) MAE. SHAP value analysis indicated that prior FVC measurements, diffusion of carbon monoxide (DLCO) values, skin involvement, age, anti-centromere positivity, dyspnea and CRP-elevation contributed most to deep-learning-based FVC predictions. Regarding uncertainty quantification, ANP RNN achieved 79% coverage (i.e. the model would provide uncertainty estimates that included the true future FVC value in 79 out of 100 predictions) out of the box, and 90% using an additional conformal prediction module with a corresponding Winkler score of 892 (indicating the width of the uncertainty estimate plus penalty for mistakes), smaller than any other model at the same coverage level. We further demonstrate how the data abstraction provided by the ANP RNN model (embeddings) allows for deriving similar patient trajectories (for similarity-based explanation). Conclusions: Our study demonstrates the feasibility of FVC forecasting and thus the ability to predict ILD trajectories in individual SSc patients using deep learning. We show that model predictions can be paired with uncertainty quantification and similarity-based model explainability, which are crucial elements for deploying trustworthy ML algorithms. Our study is thus an important first step towards reliable automated ILD trajectory (i.e. FVC%) prediction system with potential clinical utility.

show abstract

Vision transformer assisting rheumatologists in screening for capillaroscopy changes in systemic sclerosis: an artificial intelligence model

Cited by 16 publications

References 27 publications

A deep learning system for quantitative assessment of microvascular abnormalities in nailfold capillary images

A deep learning system for quantitative assessment of microvascular abnormalities in nailfold capillary images

The role of trichoscopy beyond hair and scalp diseases: A review

Predicting interstitial lung disease progression in patients with systemic sclerosis using attentive neural processes - a EUSTAR study

Contact Info

Product

Resources

About