Automating the Determination of Prostate Cancer Risk Strata From Electronic Medical Records

Abstract

Purpose Risk stratification underlies system-wide efforts to promote the delivery of appropriate prostate cancer care. Although the elements of risk stratum are available in the electronic medical record, manual data collection is resource intensive. Therefore, we investigated the feasibility and accuracy of an automated data extraction method using natural language processing (NLP) to determine prostate cancer risk stratum. Methods Manually collected clinical stage, biopsy Gleason score, and preoperative prostate-specific antigen (PSA) values from our prospective prostatectomy database were used to categorize patients as low, intermediate, or high risk by D’Amico risk classification. NLP algorithms were developed to automate the extraction of the same data points from the electronic medical record, and risk strata were recalculated. The ability of NLP to identify elements sufficient to calculate risk (recall) was calculated, and the accuracy of NLP was compared with that of manually collected data using the weighted Cohen’s κ statistic. Results Of the 2,352 patients with available data who underwent prostatectomy from 2010 to 2014, NLP identified sufficient elements to calculate risk for 1,833 (recall, 78%). NLP had a 91% raw agreement with manual risk stratification (κ = 0.92; 95% CI, 0.90 to 0.93). The κ statistics for PSA, Gleason score, and clinical stage extraction by NLP were 0.86, 0.91, and 0.89, respectively; 91.9% of extracted PSA values were within ± 1.0 ng/mL of the manually collected PSA levels. Conclusion NLP can achieve more than 90% accuracy on D’Amico risk stratification of localized prostate cancer, with adequate recall. This figure is comparable to other NLP tasks and illustrates the known trade off between recall and accuracy. Automating the collection of risk characteristics could be used to power real-time decision support tools and scale up quality measurement in cancer care.

Automating the Capture of Structured Pathology Data for Prostate Cancer Clinical Care and Research

“…27 Gregg and colleagues used NLP across clinical notes for patients with prostate cancer and were able to assign D'Amico risk groups (low, intermediate, high) with 92% accuracy. 28 Our work, using a rule-based NLP approach, achieved similar results to other nvestigators. 24,25,29 However, instead of comparing results to only manual abstraction in a test environment, we report results from a real-world clinical implementation and compare with both manual abstraction and clinician-entered SDEs at the point of care.…”

Use of Natural Language Processing to Extract Clinical Cancer Phenotypes from Electronic Medical Records

“…In research, real-world big data have great potential to improve cancer care. Gregg and colleagues present a risk stratification research for prostate cancer (85). The utilization of real-world big data is a key focus area of the NCI (86).…”

Natural Language Processing for Automated Quantification of Brain Metastases Reported in Free-Text Radiology Reports

“…NLP for the analysis of radiology reports has also been explored in the context of other cancer types, including hepatocellular carcinomas, [13][14][15][16] breast cancer, [17][18][19][20] lung cancer, [21][22][23] and other abdominal or pelvic tumors. 11,[24][25][26][27] All studies that provided sufficient insight into their modeling approach used a bag-of-words approach. To our knowledge, this study presents the first sequence-based NLP approach for analyzing free-text radiology reports in oncology patients as well as the first head-to-head comparison of sequence-based and bag-of-words models for medical text analysis.…”