Background
Patient Priorities Care (PPC) is a model of care that aligns health care recommendations with priorities of older adults who have multiple chronic conditions. Following identification of patient priorities, this information is documented in the patient’s electronic health record (EHR).
Objective
Our goal is to develop and validate a natural language processing (NLP) model that reliably documents when clinicians identify patient priorities (ie, values, outcome goals, and care preferences) within the EHR as a measure of PPC adoption.
Methods
This is a retrospective analysis of unstructured National Veteran Health Administration EHR free-text notes using an NLP model. The data were sourced from 778 patient notes of 658 patients from encounters with 144 social workers in the primary care setting. Each patient’s free-text clinical note was reviewed by 2 independent reviewers for the presence of PPC language such as priorities, values, and goals. We developed an NLP model that utilized statistical machine learning approaches. The performance of the NLP model in training and validation with 10-fold cross-validation is reported via accuracy, recall, and precision in comparison to the chart review.
Results
Of 778 notes, 589 (75.7%) were identified as containing PPC language (kappa=0.82, P<.001). The NLP model in the training stage had an accuracy of 0.98 (95% CI 0.98-0.99), a recall of 0.98 (95% CI 0.98-0.99), and precision of 0.98 (95% CI 0.97-1.00). The NLP model in the validation stage had an accuracy of 0.92 (95% CI 0.90-0.94), recall of 0.84 (95% CI 0.79-0.89), and precision of 0.84 (95% CI 0.77-0.91). In contrast, an approach using simple search terms for PPC only had a precision of 0.757.
Conclusions
An automated NLP model can reliably measure with high precision, recall, and accuracy when clinicians document patient priorities as a key step in the adoption of PPC.
Elder abuse (EA) is common and has devastating health consequences yet is rarely detected by healthcare professionals. Veterans are at high risk for EA, and the Veterans Health Administration (VHA) has experience screening for complex psychosocial phenomena including intimate partner violence. While the VHA has national policy regarding mandatory reporting of EA cases, little is known about the extent to which VHA sites currently screen for EA in a standardized fashion and what approaches are used. To address this knowledge gap, we conducted a national survey of all 170 parent station VHA medical centers from January to August of 2021. Surveys were distributed electronically to the Social Work Chief at each site, as social work is responsible for interpersonal violence response in VHA. The survey assessed the presence and characteristics of EA-specific screening practices as well as general abuse/neglect screening conducted with patients of all ages, including older adults. Follow up emails were sent to sites who reported conducting screening requesting additional details not included in the initial survey. Overall, 138 sites (81%) responded to the survey. Among respondents, 3% reported screening older adults for EA using a previously published tool, while 2% reported screening for EA with an unstudied or locally-developed tool. Forty-three percent reported doing general abuse/neglect screening using unstudied questions/tools for patients of all ages, and 41% reported no EA screening at their site. The wide variability in current EA screening practices in VHA presents an important opportunity to standardize and improve EA detection practices.
Elder abuse (EA) is common and has devastating health impacts, yet most cases go undetected limiting opportunities to intervene. Older Veterans receiving care in the Veterans Health Administration (VHA) represent a high-risk population for EA. VHA emergency department (ED) visits provide a unique opportunity to identify EA, as assessment for acute injury or illness may be the only time isolated older Veterans leave their home, but most VHA EDs do not have standardized EA assessment protocols. To address this, we assembled an interdisciplinary team of VHA social workers, physicians, nurses, intermediate care technicians (ICTs; former military medics and corpsmen who often conduct screenings in VHA EDs) and both VHA and non-VHA EA subject matter experts to adapt the Elder Mistreatment Screening and Response Tool (EM-SART) to pilot in the Louis Stokes Cleveland VA Medical Center geriatric ED (GED)program. The cornerstone of their approach is an interdisciplinary GED consultation led by ICTs and nurses who screen high-risk older Veterans for geriatric syndromes and unmet needs. The adapted EM-SART was integrated into the electronic health record and GED workflow in December 2020. By July 2022, a total of 251 Veterans were screened with nine (3.6%) positive on the prescreen and five (2%) positive on the
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.