No abstract

Key Points Question How accurate are dictated clinical documents created by speech recognition software, edited by professional medical transcriptionists, and reviewed and signed by physicians? Findings Among 217 clinical notes randomly selected from 2 health care organizations, the error rate was 7.4% in the version generated by speech recognition software, 0.4% after transcriptionist review, and 0.3% in the final version signed by physicians. Among the errors at each stage, 15.8%, 26.9%, and 25.9% involved clinical information, and 5.7%, 8.9%, and 6.4% were clinically significant, respectively. Meaning An observed error rate of more than 7% in speech recognition–generated clinical documents demonstrates the importance of manual editing and review.

Objective The study sought to review recent literature regarding use of speech recognition (SR) technology for clinical documentation and to understand the impact of SR on document accuracy, provider efficiency, institutional cost, and more. Materials and Methods We searched 10 scientific and medical literature databases to find articles about clinician use of SR for documentation published between January 1, 1990, and October 15, 2018. We annotated included articles with their research topic(s), medical domain(s), and SR system(s) evaluated and analyzed the results. Results One hundred twenty-two articles were included. Forty-eight (39.3%) involved the radiology department exclusively and 10 (8.2%) involved emergency medicine; 10 (8.2%) mentioned multiple departments. Forty-eight (39.3%) articles studied productivity; 20 (16.4%) studied the effect of SR on documentation time, with mixed findings. Decreased turnaround time was reported in all 19 (15.6%) studies in which it was evaluated. Twenty-nine (23.8%) studies conducted error analyses, though various evaluation metrics were used. Reported percentage of documents with errors ranged from 4.8% to 71%; reported word error rates ranged from 7.4% to 38.7%. Seven (5.7%) studies assessed documentation-associated costs; 5 reported decreases and 2 reported increases. Many studies (44.3%) used products by Nuance Communications. Other vendors included IBM (9.0%) and Philips (6.6%); 7 (5.7%) used self-developed systems. Conclusion Despite widespread use of SR for clinical documentation, research on this topic remains largely heterogeneous, often using different evaluation metrics with mixed findings. Further, that SR-assisted documentation has become increasingly common in clinical settings beyond radiology warrants further investigation of its use and effectiveness in these settings.

This study developed an innovative natural language processing algorithm to automatically identify heart failure (HF) patients with ineffective self-management status (in the domains of diet, physical activity, medication adherence, and adherence to clinician appointments) from narrative discharge summary notes. We also analyzed the association between self-management status and preventable 30-day hospital readmissions. Our natural language system achieved relatively high accuracy ( F-measure = 86.3%; precision = 95%; recall = 79.2%) on a testing sample of 300 notes annotated by two human reviewers. In a sample of 8,901 HF patients admitted to our healthcare system, 14.4% ( n = 1,282) had documentation of ineffective HF self-management. Adjusted regression analyses indicated that presence of any skill-related self-management deficit (odds ratio [OR] = 1.3, 95% confidence interval [CI] = [1.1, 1.6]) and non-specific ineffective self-management (OR = 1.5, 95% CI = [1.2, 2]) was significantly associated with readmissions. We have demonstrated the feasibility of identifying ineffective HF self-management from electronic discharge summaries with natural language processing.

BackgroundTime in the therapeutic range (TTR) is associated with the effectiveness and safety of vitamin K antagonist (VKA) therapy. To optimize prescribing of VKA, we aimed to develop and validate a prediction model for TTR in older adults taking VKA for nonvalvular atrial fibrillation and venous thromboembolism.Methods and ResultsThe study cohort comprised patients aged ≥65 years who were taking VKA for atrial fibrillation or venous thromboembolism and who were identified in the 2 US electronic health record databases linked with Medicare claims data from 2007 through 2014. With the predictors identified from a systematic review and clinical knowledge, we built a prediction model for TTR, using one electronic health record system as the training set and the other as the validation set. We compared the performance of the new models to that of a published prediction score for TTR, SAMe‐TT 2R2. Based on 1663 patients in the training set and 1181 in the validation set, our optimized score included 42 variables and the simplified model included 7 variables, abbreviated as PROSPER (Pneumonia, Renal dysfunction, Oozing blood [prior bleeding], Staying in hospital ≥7 days, Pain medication use, no Enhanced [structured] anticoagulation services, Rx for antibiotics). The PROSPER score outperformed SAMe‐TT 2R2 when predicting both TTR ≥70% (area under the receiver operating characteristic curve 0.67 versus 0.55) and the thromboembolic and bleeding outcomes (area under the receiver operating characteristic curve 0.62 versus 0.52).ConclusionsOur geriatric TTR score can be used as a clinical decision aid to select appropriate candidates to receive VKA therapy and as a research tool to address confounding and treatment effect heterogeneity by anticoagulation quality.