Enhancing Case Capture, Quality, and Completeness of Primary Melanoma Pathology Records via Natural Language Processing

Abstract

PURPOSE Medical records contain a wealth of useful, informative data points valuable for clinical research. Most data points are stored in semistructured or unstructured legacy documents and require manual data abstraction into a structured format to render the information more readily accessible, searchable, and generally analysis ready. The substantial labor needed for this can be cost prohibitive, particularly when dealing with large patient cohorts. METHODS To establish a high-throughput approach to data abstraction, we developed a novel framework using natural language processing (NLP) and a decision-rules algorithm to extract, transform, and load (ETL) melanoma primary pathology features from pathology reports in an institutional legacy electronic medical record system into a structured database. We compared a subset of these data with a manually curated data set comprising the same patients and developed a novel scoring system to assess confidence in records generated by the algorithm, thus obviating manual review of high-confidence records while flagging specific, low-confidence records for review. RESULTS The algorithm generated 368,624 individual melanoma data points comprising 16 primary tumor prognostic factors and metadata from 23,039 patients. From these data points, a subset of 147,872 was compared with an existing, manually abstracted data set, demonstrating an exact or synonymous match between 90.4% of all data points. Additionally, the confidence-scoring algorithm demonstrated an error rate of only 3.7%. CONCLUSION Our NLP platform can identify and abstract melanoma primary prognostic factors with accuracy comparable to that of manual abstraction (< 5% error rate), with vastly greater efficiency. Principles used in the development of this algorithm could be expanded to include other melanoma-specific data points as well as disease-agnostic fields and further enhance capture of essential elements from nonstructured data.

A Novel Multiple Risk Score Model for Prediction of Long-Term Ischemic Risk in Patients With Coronary Artery Disease Undergoing Percutaneous Coronary Intervention: Insights From the I-LOVE-IT 2 Trial

“…However, with the improvement of the digital hospital and laboratory information system and the advent of machine learning based on deep neural networks, an approach may be a viable solution to generate detailed data in high-volume capacity ( 35 , 36 ). It is convenient to capture all factors relating to the scores in the electronic medical records (EMR), calculate them automatically, and then quickly preset the predicting risk of patients in the system in auto ( 37 , 38 ). It should be noted that dichotomizing continuous risk scores into a regression model might not be the optimal choice, which could induce a potential risk of inaccuracy.…”

The role of machine learning in clinical research: transforming the future of evidence generation

“…An appealing application of ML, specifically NLP, to study data management is to automate data collection into case report forms, decreasing the time, expense, and potential for error associated with human data extraction, whether in prospective trials or retrospective reviews. Though this use requires overcoming variable data structures and provenances, it has shown early promise in cancer [ 43 , 44 ], epilepsy [ 30 ], and depression [ 45 ], among other areas [ 29 ]. Regardless of how data have been collected, ML can power risk-based monitoring approaches to clinical trial surveillance, enabling the prevention and/or early detection of site failure, fraud, and data inconsistencies or incompleteness that may delay database lock and subsequent analysis.…”

Assessing the Prognostic Significance of Tumor-Infiltrating Lymphocytes in Patients With Melanoma Using Pathologic Features Identified by Natural Language Processing

“…Because reviewing more than 20 000 pathology reports to generate our present study cohort would require immense amounts of time and human labor, we applied the NLP algorithm during the data set construction phase. Previously, Malke et al 22 developed an NLP platform that can identify and abstract melanoma primary prognostic factors with a less than 5% error rate compared with manual extraction, resulting in enormous improvement in efficiency. Our NLP algorithm is capable of automatically parsing pathology reports and can extract more than 97.7% of results with 100% precision, thus significantly reducing workload and allowing for large-scale patient analysis to be performed.…”

“…Natural language processing (NLP), which enables computers to process the free-text sections automatically using knowledge-based approaches and/or machine learning algorithms, has shown potential in identifying primary melanoma cases from electronic health records and extracting histopathologic characteristics from pathology reports. 22 Previous studies have demonstrated that NLP is an effective approach for case identification and may reduce the number of manual reviews in oncology 23 , 24 and other medical domains. 25 , 26 In this study, we aimed to assess the prognostic significance of TILs in patients with cutaneous melanoma using a large patient cohort with clinical and histopathologic characteristics identified using NLP techniques.…”