Information Extraction from Echocardiography Reports for a Clinical Follow-up Study—Comparison of Extracted Variables Intended for General Use in a Data Warehouse with Those Intended Specifically for the Study

Abstract

Background The interest in information extraction from clinical reports for secondary data use is increasing. But experience with the productive use of information extraction processes over time is scarce. A clinical data warehouse has been in use at our university hospital for several years, which also provides an information extraction of echocardiography reports developed for general use. Objectives This study aims to illustrate the difficulties encountered, while using data from a preexisting information extraction process for a large clinical study. To compare the data from the preexisting process with the data obtained from a specially developed process designed to improve the quality and completeness of the study data. Methods We extracted the echocardiography variables for 440 patients from the general-use information extraction of the data warehouse (678 reports). Then we developed an information extraction process for the same variables but specifically for this study, with the aim to extract as much information as possible from the text. The extracted data of both processes were compared with a newly created gold standard defined by a cardiologist with long-standing experience in heart failure. Results Among 57 echocardiography variables considered relevant for the study, 50 were documented in the routine text reports and could be extracted. Twenty of the required variables were not provided by the general-use extraction process, some others were not provided correctly. The median macro F1-score (precision, recall) across the 30 variables for which values were extracted was 0.81 (0.94, 0.77). Across all 50 variables, as relevant for the study, median macro F1-score was only 0.49 (0.56, 0.46). Employing the study-specific approach considerably improved the quality and completeness of the variables, resulting in F1-scores of 0.97 (0.98, 0.96) across all variables. Conclusion Data from information extractions can be used for large clinical studies. However, preexisting information extraction processes should be treated with caution, as the time and effort spent defining each variable in the information extraction process may not be clear.

Sex‐specific bimodal clustering of left ventricular ejection fraction in patients with acute heart failure

“…LVEF was extracted from the clinical report by automated information extraction. 4 We first estimated the distribution of LVEF using a histogram estimator. Bin width was selected by minimizing the cross-validated empirical risk.…”

Automatic extraction of 12 cardiovascular concepts from German discharge letters using pre-trained language models

“… 9 – 12 Regarding medical texts from the cardiovascular domain there had been a number of publications on English data (e.g., Small et al, 13 Nath et al, 14 Patterson et al, 15 and Khalifa and Meystre 16 ) just a few studies focused on German texts. 17 , 18 Tasks performed in the cardiovascular domain range from text classification tasks 13 to concept and concept-value pair extraction tasks covering up to 10 concepts 19 , 20 (e.g. risk factors), 16 to a broad range of cardiovascular concepts (CCs) covering up to 80 different clinical values.…”

“…risk factors), 16 to a broad range of cardiovascular concepts (CCs) covering up to 80 different clinical values. 15 , 17 , 18 , 21 All of the cardiovascular-related publications used either rule- and pattern-based approaches or commercial text mining tools.…”