Deep Learning for Cancer Symptoms Monitoring on the Basis of Electronic Health Record Unstructured Clinical Notes

Abstract

PURPOSE Symptoms are vital outcomes for cancer clinical trials, observational research, and population-level surveillance. Patient-reported outcomes (PROs) are valuable for monitoring symptoms, yet there are many challenges to collecting PROs at scale. We sought to develop, test, and externally validate a deep learning model to extract symptoms from unstructured clinical notes in the electronic health record. METHODS We randomly selected 1,225 outpatient progress notes from among patients treated at the Dana-Farber Cancer Institute between January 2016 and December 2019 and used 1,125 notes as our training/validation data set and 100 notes as our test data set. We evaluated the performance of 10 deep learning models for detecting 80 symptoms included in the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework. Model performance as compared with manual chart abstraction was assessed using standard metrics, and the highest performer was externally validated on a sample of 100 physician notes from a different clinical context. RESULTS In our training and test data sets, 75 of the 80 candidate symptoms were identified. The ELECTRA-small model had the highest performance for symptom identification at the token level (ie, at the individual symptom level), with an F1 of 0.87 and a processing time of 3.95 seconds per note. For the 10 most common symptoms in the test data set, the F1 score ranged from 0.98 for anxious to 0.86 for fatigue. For external validation of the same symptoms, the note-level performance ranged from F1 = 0.97 for diarrhea and dizziness to F1 = 0.73 for swelling. CONCLUSION Training a deep learning model to identify a wide range of electronic health record–documented symptoms relevant to cancer care is feasible. This approach could be used at the health system scale to complement to electronic PROs.

Development of a keyword library for capturing PRO-CTCAE-focused “symptom talk” in oncology conversations

“…Our hope is that automating some level of symptom detection from natural conversations may help detect symptom burden sooner, contributing to targeted, earlier integration of palliative care. 40 Using conversation data may help us assess and address symptoms that go undocumented given their “expected” nature in cancer. 30 , 73 As patient-provider communication is often implicated as barring cancer pain relief, 74 involving computational methods in symptom detection could help capture symptoms that patients may not have explicit language for.…”

“…Disagreements were adjudicated via discussion with the larger study group. All words and phrases were discussed and sorted into PRO-CTCAE symptom categories (nonexclusive); following this, a priori additions were gathered from prior work on clinical notes 40 and further brainstormed as supplemental additions.…”

“…Currently, efforts to detect symptoms via computational methods are evolving. AI models are being built to summarize medical conversations, 38 and extract symptoms from clinical notes 39 , 40 ; yet, research remains limited, specific to few symptoms, and largely lacks representation of verbal patient language.…”

“…Symptom keyword libraries have so far been generated from patient-authored text 41 and written clinical notes 40 ; there is not yet a public library base grounded in patients’ verbal conversation data. A library of this sort could help researchers detect symptoms from the rich, encompassing data source of audio-recorded clinical encounters, which are increasingly available with the rise of Telehealth and offer a high-impact setting for computational tools to record and summarize symptoms both in-the-moment and longitudinally.…”

Opportunities for computational tools in palliative care: Supporting patient needs and lowering burden

“…By improving the summarization, searchability, and readability of clinical information from the electronic health record, natural language processing tools can assist providers in highlighting key information regarding patient care, as well as to more easily coordinate across care teams. For example, the Lindvall lab is currently developing tools to detect symptom content from unstructured data; MLSym is a project focused on capturing and summarizing symptoms documented within the electronic health record 6 and ongoing work seeks to capture the same content from clinical conversations. These and similar tools could help determine what symptoms were discussed or documented in a given week.…”