Syntactic and semantic errors in radiology reports associated with speech recognition software

Ringler, Michael D.; Goss, Brian C.; Bartholmai, Brian J.

doi:10.1177/1460458215613614

Cited by 25 publications

(14 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speech recognition errors are well known since the introduction of this technique and are more common compared with manual report transcription [18]. Errors vary in importance, ranging from trivial spelling errors to alterations of meaning and possible interpretation of reports [19]. This affects all medical subspecialties using speech recognition with Table 3 Overview of the top 5 most frequently added words to the findings section of analyzed radiology reports.…”

Section: Discussionmentioning

confidence: 99%

Revealing the most common reporting errors through data mining of the report proofreading process

et al. 2020

View full text Add to dashboard Cite

Objectives To investigate the most common errors in residents’ preliminary reports, if structured reporting impacts error types and frequencies, and to identify possible implications for resident education and patient safety. Material and methods Changes in report content were tracked by a report comparison tool on a word level and extracted for 78,625 radiology reports dictated from September 2017 to December 2018 in our department. Following data aggregation according to word stems and stratification by subspecialty (e.g., neuroradiology) and imaging modality, frequencies of additions/deletions were analyzed for findings and impression report section separately and compared between subgroups. Results Overall modifications per report averaged 4.1 words, with demonstrably higher amounts of changes for cross-sectional imaging (CT: 6.4; MRI: 6.7) than non-cross-sectional imaging (radiographs: 0.2; ultrasound: 2.8). The four most frequently changed words (right, left, one, and none) remained almost similar among all subgroups (range: 0.072–0.117 per report; once every 9–14 reports). Albeit representing only 0.02% of analyzed words, they accounted for up to 9.7% of all observed changes. Subspecialties solely using structured reporting had substantially lower change ratios in the findings report section (mean: 0.2 per report) compared with prose-style reporting subspecialties (mean: 2.0). Relative frequencies of the most changed words remained unchanged. Conclusion Residents’ most common reporting errors in all subspecialties and modalities are laterality discriminator confusions (left/right) and unnoticed descriptor misregistration by speech recognition (one/none). Structured reporting reduces overall error rates, but does not affect occurrence of the most common errors. Increased error awareness and measures improving report correctness and ensuring patient safety are required. Key Points • The two most common reporting errors in residents’ preliminary reports are laterality discriminator confusions (left/right) and unnoticed descriptor misregistration by speech recognition (one/none). • Structured reporting reduces the overall the error frequency in the findings report section by a factor of 10 (structured reporting: mean 0.2 per report; prose-style reporting: 2.0) but does not affect the occurrence of the two major errors. • Staff radiologist review behavior noticeably differs between radiology subspecialties.

show abstract

Section: Discussionmentioning

confidence: 99%

Revealing the most common reporting errors through data mining of the report proofreading process

et al. 2020

View full text Add to dashboard Cite

show abstract

“…45 Leaving a misinterpreted word uncorrected may result in unclear documentation, embarrassing errors, and patient safety issues. [45][46][47][48][49][50][51][52][53][54] The provider burden to modify misunderstood words has been a cause of dissatisfaction with SR technology. Because computers have limited capabilities to format and correct grammar, providers spend more time correcting mistakes using computer transcription than with human transcription.…”

Section: Speechmentioning

confidence: 99%

Electronic Health Record Interactions through Voice: A Review

et al. 2018

View full text Add to dashboard Cite

show abstract

“…More recent studies performed with state-of-the-art dictation systems also demonstrate high error rates. A 2015 review of 213,977 radiology reports at the Mayo Clinic found a 9.7% error rate, ranging from a high of a 19.7% in neuroradiology to a low of 3.2% in chest plain films (6). Most significantly, nearly 20% of these reports containing errors were found to contain a clinically material error (6).…”

Section: Introductionmentioning

confidence: 99%

Detecting insertion, substitution, and deletion errors in radiology reports using neural sequence-to-sequence models

Zech

Forde²,

Titano

et al. 2019

Ann. Transl. Med

View full text Add to dashboard Cite

Background: Errors in grammar, spelling, and usage in radiology reports are common. To automatically detect inappropriate insertions, deletions, and substitutions of words in radiology reports, we proposed using a neural sequence-to-sequence (seq2seq) model. Methods: Head CT and chest radiograph reports from Mount Sinai Hospital (MSH) (n=61,722 and 818,978, respectively), Mount Sinai Queens (MSQ) (n=30,145 and 194,309, respectively) and MIMIC-III (n=32,259 and 54,685) were converted into sentences. Insertions, substitutions, and deletions of words were randomly introduced. Seq2seq models were trained using corrupted sentences as input to predict original uncorrupted sentences. Three models were trained using head CTs from MSH, chest radiographs from MSH, and head CTs from all three collections. Model performance was assessed across different sites and modalities. A sample of original, uncorrupted sentences were manually reviewed for any error in syntax, usage, or spelling to estimate real-world proofreading performance of the algorithm. Results: Seq2seq detected 90.3% and 88.2% of corrupted sentences with 97.7% and 98.8% specificity in same-site, same-modality test sets for head CTs and chest radiographs, respectively. Manual review of original, uncorrupted same-site same-modality head CT sentences demonstrated seq2seq positive predictive value (PPV) 0.393 (157/400; 95% CI, 0.346-0.441) and negative predictive value (NPV) 0.986 (789/800; 95% CI, 0.976-0.992) for detecting sentences containing real-world errors, with estimated sensitivity of 0.389 (95% CI, 0.267-0.542) and specificity 0.986 (95% CI, 0.985-0.987) over n=86,211 uncorrupted training examples. Conclusions: Seq2seq models can be highly effective at detecting erroneous insertions, deletions, and substitutions of words in radiology reports. To achieve high performance, these models require site-and modality-specific training examples. Incorporating additional targeted training data could further improve performance in detecting real-world errors in reports.

show abstract

Syntactic and semantic errors in radiology reports associated with speech recognition software

Cited by 25 publications

References 22 publications

Revealing the most common reporting errors through data mining of the report proofreading process

Revealing the most common reporting errors through data mining of the report proofreading process

Electronic Health Record Interactions through Voice: A Review

Detecting insertion, substitution, and deletion errors in radiology reports using neural sequence-to-sequence models

Contact Info

Product

Resources

About