The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital

Abstract

BackgroundSpeech recognition (SR) technology, the process whereby spoken words are converted to digital text, has been used in radiology reporting since 1981. It was initially anticipated that SR would dominate radiology reporting, with claims of up to 99% accuracy, reduced turnaround times and significant cost savings. However, expectations have not yet been realised. The limited data available suggest SR reports have significantly higher levels of inaccuracy than traditional dictation transcription (DT) reports, as well as incurring greater aggregate costs.There has been little work on the clinical significance of such errors, however, and little is known of the impact of reporter seniority on the generation of errors, or the influence of system familiarity on reducing error rates.Furthermore, there have been conflicting findings on the accuracy of SR amongst users with English as first- and second-language respectively.MethodsThe aim of the study was to compare the accuracy of SR and DT reports in a resource-limited setting. The first 300 SR and the first 300 DT reports generated during March 2010 were retrieved from the hospital’s PACS, and reviewed by a single observer. Text errors were identified, and then classified as either clinically significant or insignificant based on their potential impact on patient management. In addition, a follow-up analysis was conducted exactly 4 years later.ResultsOf the original 300 SR reports analysed, 25.6% contained errors, with 9.6% being clinically significant. Only 9.3% of the DT reports contained errors, 2.3% having potential clinical impact. Both the overall difference in SR and DT error rates, and the difference in ‘clinically significant’ error rates (9.6% vs. 2.3%) were statistically significant. In the follow-up study, the overall SR error rate was strikingly similar at 24.3%, 6% being clinically significant.Radiologists with second-language English were more likely to generate reports containing errors, but level of seniority had no bearing.ConclusionSR technology consistently increased inaccuracies in Tygerberg Hospital (TBH) radiology reports, thereby potentially compromising patient care. Awareness of increased error rates in SR reports, particularly amongst those transcribing in a second-language, is important for effective implementation of SR in a multilingual healthcare environment.

Revealing the most common reporting errors through data mining of the report proofreading process

“…Speech recognition errors are well known since the introduction of this technique and are more common compared with manual report transcription [18]. Errors vary in importance, ranging from trivial spelling errors to alterations of meaning and possible interpretation of reports [19].…”

Electronic Health Record Interactions through Voice: A Review

“…45 Leaving a misinterpreted word uncorrected may result in unclear documentation, embarrassing errors, and patient safety issues. [45][46][47][48][49][50][51][52][53][54] The provider burden to modify misunderstood words has been a cause of dissatisfaction with SR technology. Because computers have limited capabilities to format and correct grammar, providers spend more time correcting mistakes using computer transcription than with human transcription.…”

Accuracy of Speech Recognition System’s Medical Report and Physicians' Experience in Hospitals

“…Successful use of this technology has been reported in radiology [1,2]. These systems are designed to turn speaker's speech into text and eliminate the need for human transcription and dictation [1,[3][4][5]. Primary systems have multiple limitations such as limited vocabulary banks, time consumption, low speed, non-userfriendliness and most importantly, separate speech.…”