Automated quantitative MRI volumetry reports support diagnostic interpretation in dementia: a multi-rater, clinical accuracy study

Abstract

Objectives We examined whether providing a quantitative report (QReport) of regional brain volumes improves radiologists’ accuracy and confidence in detecting volume loss, and in differentiating Alzheimer’s disease (AD) and frontotemporal dementia (FTD), compared with visual assessment alone. Methods Our forced-choice multi-rater clinical accuracy study used MRI from 16 AD patients, 14 FTD patients, and 15 healthy controls; age range 52–81. Our QReport was presented to raters with regional grey matter volumes plotted as percentiles against data from a normative population (n = 461). Nine raters with varying radiological experience (3 each: consultants, registrars, ‘non-clinical image analysts’) assessed each case twice (with and without the QReport). Raters were blinded to clinical and demographic information; they classified scans as ‘normal’ or ‘abnormal’ and if ‘abnormal’ as ‘AD’ or ‘FTD’. Results The QReport improved sensitivity for detecting volume loss and AD across all raters combined (p = 0.015* and p = 0.002*, respectively). Only the consultant group’s accuracy increased significantly when using the QReport (p = 0.02*). Overall, raters’ agreement (Cohen’s κ) with the ‘gold standard’ was not significantly affected by the QReport; only the consultant group improved significantly (κs 0.41➔0.55, p = 0.04*). Cronbach’s alpha for interrater agreement improved from 0.886 to 0.925, corresponding to an improvement from ‘good’ to ‘excellent’. Conclusion Our QReport referencing single-subject results to normative data alongside visual assessment improved sensitivity, accuracy, and interrater agreement for detecting volume loss. The QReport was most effective in the consultants, suggesting that experience is needed to fully benefit from the additional information provided by quantitative analyses. Key Points • The use of quantitative report alongside routine visual MRI assessment improves sensitivity and accuracy for detecting volume loss and AD vs visual assessment alone. • Consultant neuroradiologists’ assessment accuracy and agreement (kappa scores) significantly improved with the use of quantitative atrophy reports. • First multi-rater radiological clinical evaluation of visual quantitative MRI atrophy report for use as a diagnostic aid in dementia.

Technical and clinical validation of commercial automated volumetric MRI tools for dementia diagnosis—a systematic review

“…These patient populations may have more subtle patterns of atrophy and QReports are likely to provide the greatest benefit to raters by flagging patients who require more regular follow-ups and reducing inter-rater variability. The results of diagnostic accuracy studies are ideally published in peer-reviewed journals [19][20][21][116][117][118]. Several companies provide lists of publications on their website.…”

“…Volumetry is widely used in the research setting and has been used to effectively index morphological change from a variety of clinical interventions in phased and randomized controlled trials [9][10][11][12][13][14][15][16][17]. Quantitative volumetric reporting tools (QReports), which automatically quantify an individual patient's regional brain volumes and compare them to healthy, age-specific reference populations, can potentially help neuroradiologists interpret the severity and distribution of brain atrophy and contextualize their findings by referencing normative brain volumes in healthy populations [18][19][20][21][22][23]. The limitations of routine visual assessment reveal the area of clinical need in which such tools can be integrated.…”

“…The limitations of routine visual assessment reveal the area of clinical need in which such tools can be integrated. Quantitative assessment of MRIs can provide more objective imaging biomarkers, contribute to the earlier identification of atrophy [24][25][26] and might improve the accuracy of radiological diagnosis of Alzheimer's disease (AD) and other subtypes of dementia [18][19][20][21][22][23]. However, there remains a large discrepancy between the use of visual rating scales and the availability of QReports in the clinic.…”

Inter- and Intra-Scanner Variability of Automated Brain Volumetry on Three Magnetic Resonance Imaging Systems in Alzheimer’s Disease and Controls

“…Therefore, recent developments in the field of imaging artificial intelligence (AI) have enabled the automatic extraction of clinically relevant measures from brain MRI scans (Niemantsverdriet et al, 2018;Struyfs et al, 2020;Wittens et al, 2021). Thereupon, it has been shown that automated volumetry, combined with the expertise of radiologists, can improve the sensitivity and specificity of assessing AD-related atrophy (Pemberton et al, 2021). Since combining modern AI technology with radiological expertise has the potential to detect and monitor abnormalities more accurately, a strict validation of AI tools is necessary to assess the validity in a clinical setting.…”

Editorial for “MRI Findings of Arachnoiditis, Revisited. Is Classification Needed?”

“…Neuroradiological studies involving raters from different specialities and experience levels have been on the rise in recent years. [2][3][4][5][6][7] Since the advent of MRI, lumbosacral arachnoiditis classification has been based on a three-group system pertaining to the T2-weighted appearance of nerve roots: group 1, the nerve roots clump in the center of the thecal sac; in group 2, the nerve roots adhere to the periphery of the thecal sac; in group 3, the nerve roots form a mass in the thecal sac. However, this paper, the first in the literature to examine radiological classification of lumbosacral arachnoiditis, provides further evidence of heterogeneous imaging findings and encourages moving beyond the traditional three-group classification.…”