Retrospective analysis and prospective validation of an AI-based software for intracranial haemorrhage detection at a high-volume trauma centre

Abstract

Rapid detection of intracranial haemorrhage (ICH) is crucial for assessing patients with neurological symptoms. Prioritising these urgent scans for reporting presents a challenge for radiologists. Artificial intelligence (AI) offers a solution to enable radiologists to triage urgent scans and reduce reporting errors. This study aims to evaluate the accuracy of an ICH-detection AI software and whether it benefits a high-volume trauma centre in terms of triage and reducing diagnostic errors. A peer review of head CT scans performed prior to the implementation of the AI was conducted to identify the department’s current miss-rate. Once implemented, the AI software was validated using CT scans performed over one month, and was reviewed by a neuroradiologist. The turn-around-time was calculated as the time taken from scan completion to report finalisation. 2916 head CT scans and reports were reviewed as part of the audit. The AI software flagged 20 cases that were negative-by-report. Two of these were true-misses that had no follow-up imaging. Both patients were followed up and exhibited no long-term neurological sequelae. For ICH-positive scans, there was an increase in TAT in the total sample (35.6%), and a statistically insignificant decrease in TAT in the emergency (− 5.1%) and outpatient (− 14.2%) cohorts. The AI software was tested on a sample of real-world data from a high-volume Australian centre. The diagnostic accuracy was comparable to that reported in literature. The study demonstrated the institution’s low miss-rate and short reporting time, therefore any improvements from the use of AI would be marginal and challenging to measure.

Efficiency, accuracy, and health professional's perspectives regarding artificial intelligence in radiology practice: A scoping review

“…[28] and Zia et al. [38] reported potential biases in reporting time when the AI system used in radiology practices is not integrated into the clinical reporting system. Zia's results showed significantly prolonged reading times using AI, which was installed on personal computers that were not integrated with the radiology reporting system.…”

“…Overall, most observational studies (5/7, 71.43%) were judged to have a low ROB in all aspects; however, two studies (28.57%) had a medium ROB. Seven articles investigated the effect of AI on radiologists' reading time in controlled experiments; those studies used AI to identify lung-related abnormalities [28][29][30][31], breast cancer [7,35], and intracranial hemorrhage [38]. Most studies involved two to six radiologists, except for a study by Zia et al, in which an entire radiology department was surveyed but the number of respondents was not disclosed [38].…”

“…Seven articles investigated the effect of AI on radiologists' reading time in controlled experiments; those studies used AI to identify lung-related abnormalities [28][29][30][31], breast cancer [7,35], and intracranial hemorrhage [38]. Most studies involved two to six radiologists, except for a study by Zia et al, in which an entire radiology department was surveyed but the number of respondents was not disclosed [38]. Calisto et al surveyed 45 physicians, with each reading only three cases [35].…”

“…Calisto et al found no substantial reduction in reading time for cases with medium or high severity but observed a reduction for low-severity cases [35]. Zia et al found longer reading times with use of AI in the inpatient department but observed significant time savings for cases involving intracranial hemorrhage in the emergency and outpatient departments [38].…”

“…The level of AI integration in radiology practices and their underlying electronic systems was found to be a key factor in increasing the impact of AI on the efficiency of such practices. Ahn et al [28] and Zia et al [38] reported potential biases in reporting time when the AI system used in radiology practices is not integrated into the clinical reporting system. Zia's results showed significantly prolonged reading times using AI, which was installed on personal computers that were not integrated with the radiology reporting system.…”