Natural Language Processing to Identify Advance Care Planning Documentation in a Multisite Pragmatic Clinical Trial

Deng, Chih-Ying; Moseley, Edward T.; Agaronnik, Nicole; El‐Jawahri, Areej; Lakin, Joshua R.; Volandes, Angelo E.

doi:10.1016/j.jpainsymman.2021.06.025

Cited by 44 publications

(39 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…31,43,44 Interestingly, studies have shown that NLP models can outperform humans in identifying text-based data. 13,45 In conclusion, we demonstrated that NLP methods can be applied to EHRs for extraction of symptoms at scale. The use of the PRO-CTCAE framework to guide training of deep learning ensures that the model captures a variety of symptoms considered to be most clinically meaningful in the oncology context.…”

Section: Discussionmentioning

confidence: 72%

See 1 more Smart Citation

Deep Learning for Cancer Symptoms Monitoring on the Basis of Electronic Health Record Unstructured Clinical Notes

Deng

Agaronnik

Kwok

et al. 2022

JCO Clinical Cancer Informatics

Self Cite

View full text Add to dashboard Cite

PURPOSE Symptoms are vital outcomes for cancer clinical trials, observational research, and population-level surveillance. Patient-reported outcomes (PROs) are valuable for monitoring symptoms, yet there are many challenges to collecting PROs at scale. We sought to develop, test, and externally validate a deep learning model to extract symptoms from unstructured clinical notes in the electronic health record. METHODS We randomly selected 1,225 outpatient progress notes from among patients treated at the Dana-Farber Cancer Institute between January 2016 and December 2019 and used 1,125 notes as our training/validation data set and 100 notes as our test data set. We evaluated the performance of 10 deep learning models for detecting 80 symptoms included in the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework. Model performance as compared with manual chart abstraction was assessed using standard metrics, and the highest performer was externally validated on a sample of 100 physician notes from a different clinical context. RESULTS In our training and test data sets, 75 of the 80 candidate symptoms were identified. The ELECTRA-small model had the highest performance for symptom identification at the token level (ie, at the individual symptom level), with an F1 of 0.87 and a processing time of 3.95 seconds per note. For the 10 most common symptoms in the test data set, the F1 score ranged from 0.98 for anxious to 0.86 for fatigue. For external validation of the same symptoms, the note-level performance ranged from F1 = 0.97 for diarrhea and dizziness to F1 = 0.73 for swelling. CONCLUSION Training a deep learning model to identify a wide range of electronic health record–documented symptoms relevant to cancer care is feasible. This approach could be used at the health system scale to complement to electronic PROs.

show abstract

Section: Discussionmentioning

confidence: 72%

“…31,43,44 Interestingly, studies have shown that NLP models can outperform humans in identifying text-based data. 13,45…”

Section: Discussionmentioning

confidence: 99%

Deep Learning for Cancer Symptoms Monitoring on the Basis of Electronic Health Record Unstructured Clinical Notes

Deng

Agaronnik

Kwok

et al. 2022

JCO Clinical Cancer Informatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The primary outcome of this trial is documentation of a goals-of-care conversation in the EHR at any time during the index hospitalisation, as ascertained by NLP-assisted review of clinical notes accumulated during that hospitalisation. Similar to our previous studies, documentation that will count towards the outcome will include a discussion with the patient regarding limitations of life-sustaining treatment, palliative care, hospice, goals-of-care, time-limited trial or surrogate decision-makers 43 54. One secondary outcome of the trial is EHR documentation reflecting the presence and content of treatment preferences relating to resuscitation, feeding tubes and dialysis.…”

Section: Methodsmentioning

confidence: 97%

Study protocol for Video Images about Decisions to Improve Ethical Outcomes with Palliative Care Educators (VIDEO-PCE): a pragmatic stepped wedge cluster randomised trial of older patients admitted to the hospital

et al. 2022

View full text Add to dashboard Cite

Introduction Despite the known benefit to patients and families, discussions about goals, values and preferences for medical care in advancing serious illness often do not occur. Many system and clinician factors, such as patient and clinician reticence and shortage of specialty palliative care teams, contribute to this lack of communication. To address this gap, we designed an intervention to promote goals-of-care conversations and palliative care referrals in the hospital setting by using trained palliative care educators and video decision aids. This paper presents the rationale, design and methods for a trial aimed at addressing barriers to goals-of-care conversations for hospitalised adults aged 65 and older and those with Alzheimer’s disease and related Dementias, regardless of age. Methods and analysis The Video Image about Decisions to Improve Ethical Outcomes with Palliative Care Educators is a pragmatic stepped wedge, cluster randomised controlled trial, which aims to improve and extend goals-of-care conversations in the hospital setting with palliative care educators trained in serious illness communication and video decision aids. The primary outcome is the proportion of patients with goals-of-care documentation in the electronic health record. We estimate that over 9000 patients will be included. Ethics and dissemination The Institutional Review Board (IRB) at Boston Medical Center will serve as the single IRB of record for all regulatory and ethical aspects of this trial. BMC Protocol Number: H-41482. Findings will be presented at national meetings and in publications. This trial is registered at ClinicalTrials.gov. Trial registration number NCT04857060 ; ClinicalTrials.gov

show abstract

“…For each NLP domain (ie, goals-of-care discussion, limitations to life-sustaining treatment), we have built a keyword library with the goal of identifying relevant documentation within clinical notes. Each keyword library will be refined and validated by the review of retrospective clinical notes in each site’s local EHRs to generate formal metrics (accuracy, sensitivity, specificity, etc) across all sites 86…”

Section: Methodsmentioning

confidence: 99%

“…Each keyword library will be refined and validated by the review of retrospective clinical notes in each site’s local EHRs to generate formal metrics (accuracy, sensitivity, specificity, etc) across all sites. 86 …”

Section: Methodsmentioning

confidence: 99%

Video Images about Decisions for Ethical Outcomes in Kidney Disease (VIDEO-KD): the study protocol for a multi-centre randomised controlled trial

et al. 2022

Self Cite

View full text Add to dashboard Cite

IntroductionOlder patients with advanced chronic kidney disease (CKD) often are inadequately prepared to make informed decisions about treatments including dialysis and cardiopulmonary resuscitation. Further, evidence shows that patients with advanced CKD do not commonly engage in advance care planning (ACP), may suffer from poor quality of life, and may be exposed to end-of-life care that is not concordant with their goals. We aim to study the effectiveness of a video intervention on ACP, treatment preferences and other patient-reported outcomes.Methods and analysisThe Video Images about Decisions for Ethical Outcomes in Kidney Disease trial is a multi-centre randomised controlled trial that will test the effectiveness of an intervention that includes a CKD-related video decision aid followed by recording personal video declarations about goals of care and treatment preferences in older adults with advancing CKD. We aim to enrol 600 patients over 5 years at 10 sites.Ethics and disseminationRegulatory and ethical aspects of this trial include a single Institutional Review Board mechanism for approval, data use agreements among sites, and a Data Safety and Monitoring Board. We intend to disseminate findings at national meetings and publish our results.Trial registration numberNCT04347629.

show abstract

Natural Language Processing to Identify Advance Care Planning Documentation in a Multisite Pragmatic Clinical Trial

Cited by 44 publications

References 33 publications

Deep Learning for Cancer Symptoms Monitoring on the Basis of Electronic Health Record Unstructured Clinical Notes

Deep Learning for Cancer Symptoms Monitoring on the Basis of Electronic Health Record Unstructured Clinical Notes

Study protocol for Video Images about Decisions to Improve Ethical Outcomes with Palliative Care Educators (VIDEO-PCE): a pragmatic stepped wedge cluster randomised trial of older patients admitted to the hospital

Video Images about Decisions for Ethical Outcomes in Kidney Disease (VIDEO-KD): the study protocol for a multi-centre randomised controlled trial

Contact Info

Product

Resources

About