Automatic Extraction of Social Determinants of Health from Medical Notes of Chronic Lower Back Pain Patients

Abstract

Background. Adverse social determinants of health (SDoH), or social risk factors, such as food insecurity and housing instability, are known to contribute to poor health outcomes and inequities. Our ability to study these linkages is limited because SDoH information is more frequently documented in free-text clinical notes than structured data fields. To overcome this challenge, there is a growing push to develop techniques for automated extraction of SDoH. In this study, we explored natural language processing (NLP) and inference (NLI) methods to extract SDoH information from clinical notes of patients with chronic low back pain (cLBP), to enhance future analyses of the associations between SDoH and low back pain outcomes and disparities. Methods. Clinical notes (n=1,576) for patients with cLBP (n=386) were annotated for seven SDoH domains: housing, food, transportation, finances, insurance coverage, marital and partnership status, and other social support, resulting in 626 notes with at least one annotated entity for 364 patients. We additionally labelled pain scores, depression, and anxiety. We used a two-tier taxonomy with these 10 first-level ontological classes and 68 second-level ontological classes. We developed and validated extraction systems based on both rule-based and machine learning approaches. As a rule-based approach, we iteratively configured a clinical Text Analysis and Knowledge Extraction System (cTAKES) system. We trained two machine learning models (based on convolutional neural network (CNN) and RoBERTa transformer), and a hybrid system combining pattern matching and bag-of-words models. Additionally, we evaluated a RoBERTa based entailment model as an alternative technique of SDoH detection in clinical texts. We used a model previously trained on general domain data without additional training on our dataset. Results. Four annotators achieved high agreement (average kappa=95%, F1=91.20%). Annotation frequency varied significantly dependent on note type. By tuning cTAKES, we achieved a performance of F1=47.11% for first-level classes. For most classes, the machine learning RoBERTa-based NER model performed better (first-level F1=84.35%) than other models within the internal test dataset. The hybrid system on average performed slightly worse than the RoBERTa NER model (first-level F1=80.27%), matching or outperforming the former in terms of recall. Using an out-of-the-box entailment model, we detected many but not all challenging wordings missed by other models, reaching an average F1 of 76.04%, while matching and outperforming the tested NER models in several classes. Still, the entailment model may be sensitive to hypothesis wording and may require further fine tuning. Conclusion. This study developed a corpus of annotated clinical notes covering a broad spectrum of SDoH classes. This corpus provides a basis for training machine learning models and serves as a benchmark for predictive models for named entity recognition for SDoH and knowledge extraction from clinical texts.

Discovering Social Determinants of Health from Case Reports using Natural Language Processing: Algorithmic Development and Validation

“…In previous works, SDOH have been extracted from clinical data using different methods. These state-of-the-art methods can be categorized into conventional methods like regular expressions, dictionaries or rule-based like cTAKES 33,34 , or deep neural networks like CNN, LSTMs 35 and the latest Transformer based methods 34 . Surprisingly, in the large training sets, language model-based representations outperformed other trained neural nets.…”

“…Figure 6: COVID-19 hospitalization by race and ethnicity.DISCUSSIONPrevious works: Previous works have extracted SDOH information from clinical data using different methods such as regular expressions, dictionaries, rule-based methods like cTAKES[41,42], and deep neural networks like CNNs, LSTMs[43], and Transformer-based methods[42]. Language model-based representations have been found to perform well, especially with large training sets.…”

Large Language Models for Social Determinants of Health Information Extraction from Clinical Notes – A Generalizable Approach across Institutions

“…Firstly, many studies have focused on a limited set of SDoH factors. The review [22] revealed that among the 82 methods examined, only three SDoH factors were commonly addressed: smoking status (27 methods), substance use (21), and homelessness (20). Other critical factors such as education, insurance status, and broader social issues are still in the developmental stage.…”

“…Notably, Patra et al conducted a comprehensive review of 82 NLP methods aimed at identifying SDoH [22]. These methods span various approaches, from rule-based to deep learning-based methods, presenting the identification of SDoH as either a classification problem [16 18 19] or a named entity recognition (NER) problem [15 20 21]. For example, Stemerman et al [18] designed a multi-label classifier to identify six SDoH categories within sentences extracted from clinical notes sourced from the University of North Carolina’s clinical data warehouse.…”

“…The approaches range from utilizing rule-based techniques to deep-learning techniques and more recently large language models (LLMs) [16 17]. The problem is often formulated as a classification [16 18 19], entity recognition [20 21] or event extraction problem [15]. Majority of the existing work is limited to a single dataset from a specific institution or domain.…”