BACKGROUND Neuroimaging is the gold standard diagnostic modality for all suspected stroke patients. However, the unstructured nature of imaging reports remains a major challenge to extracting useful information from electronic health records (EHR) systems. Despite the increasing adoption of natural language processing (NLP) for radiology reports, information extraction for many stroke imaging features has not been systematically evaluated. OBJECTIVE In this study, we propose an NLP pipeline, which adopts the state-of-the-art ClinicalBERT model with domain-specific pre-training to extract 13 stroke imaging features from head computed tomography (CT) imaging notes. METHODS We utilized the model to generate structured datasets with information on the presence or absence of common stroke features for 24,924 stroke patients. We compared the survival characteristics of patients with and without features of severe stroke (midline shift, perihematomal edema, or mass effect) using the Kaplan-Meier curve and log-rank test. RESULTS Pre-trained on 82,073 head CT notes with 61 million words and fine-tuned on 200 annotated notes, our HeadCT_BERT model achieved an average Area Under Receiver Operating Characteristic curve (AUROC) of 0.9831, F1 score of 0.8683, and accuracy of 97%. Among patients with acute ischemic stroke, admissions with any severe stroke feature in initial imaging notes were associated with lower probability of survival (P-value < .001). CONCLUSIONS Our proposed NLP pipeline achieved high performance and has the potential to improve medical research and patient safety.

Background Neuroimaging is the gold-standard diagnostic modality for all patients suspected of stroke. However, the unstructured nature of imaging reports remains a major challenge to extracting useful information from electronic health records systems. Despite the increasing adoption of natural language processing (NLP) for radiology reports, information extraction for many stroke imaging features has not been systematically evaluated. Objective In this study, we propose an NLP pipeline, which adopts the state-of-the-art ClinicalBERT model with domain-specific pretraining and task-oriented fine-tuning to extract 13 stroke features from head computed tomography imaging notes. Methods We used the model to generate structured data sets with information on the presence or absence of common stroke features for 24,924 patients with strokes. We compared the survival characteristics of patients with and without features of severe stroke (eg, midline shift, perihematomal edema, or mass effect) using the Kaplan-Meier curve and log-rank tests. Results Pretrained on 82,073 head computed tomography notes with 13.7 million words and fine-tuned on 200 annotated notes, our HeadCT_BERT model achieved an average area under receiver operating characteristic curve of 0.9831, F1-score of 0.8683, and accuracy of 97%. Among patients with acute ischemic stroke, admissions with any severe stroke feature in initial imaging notes were associated with a lower probability of survival (P<.001). Conclusions Our proposed NLP pipeline achieved high performance and has the potential to improve medical research and patient safety.