Systematic Evaluation of Common Natural Language Processing Techniques to Codify Clinical Notes

Abstract

Proper codification of medical diagnoses and procedures is essential for optimized health care management, quality improvement, research, and reimbursement tasks within large healthcare systems. Assignment of diagnostic or procedure codes is a tedious manual process, often prone to human error. Natural Language Processing (NLP) have been suggested to facilitate these manual codification process. Yet, little is known on best practices to utilize NLP for such applications. Here we comprehensively assessed the performance of common NLP techniques to predict current procedural terminology (CPT) from operative notes. CPT codes are commonly used to track surgical procedures and interventions and are the primary means for reimbursement. The direct links between operative notes and CPT codes makes them a perfect vehicle to test the feasibility and performance of NLP for clinical codification. Our analysis of 100 most common musculoskeletal CPT codes suggest that traditional approaches (i.e., TF-IDF) can outperform resource intensive approaches like BERT, in addition to providing interpretability which can be very helpful and even crucial in the clinical domain. We also proposed a complexity measure to quantify the complexity of a classification task and how this measure could influence the effect of dataset size on model's performance. Finally, we provide preliminary evidence that NLP can help minimize the codification error, including mislabeling due to human error.

Building Large-Scale Registries from Unstructured Clinical Notes using a Low-Resource Natural Language Processing Pipeline

“…BERT [16] is a State-of-the-Art NLP model which has been extensively used in the healthcare domain [7,[24][25][26]. The self-attention mechanisms inherent to this model calculate the relationships between all pairs of tokens, allowing the model to identify the most informative words and phrases in each document.…”

“…Given the rapidly increasing amount of unstructured data, there is increasing interest in using natural language processing (NLP) and machine learning (ML) in registry building as an alternative to traditional methods (e.g., manual chart reviews). Notably, up to 80% of the electronic health record consists of unstructured data, providing an opportunity for NLP to become an invaluable tool to automate the processing and characterization of clinical texts into cohort-specific registries [6][7][8][9][10]. Additionally, automating the process of analyzing clinical notes and building registries can help reduce human error [7].…”

“…Notably, up to 80% of the electronic health record consists of unstructured data, providing an opportunity for NLP to become an invaluable tool to automate the processing and characterization of clinical texts into cohort-specific registries [6][7][8][9][10]. Additionally, automating the process of analyzing clinical notes and building registries can help reduce human error [7].…”

Natural language processing: using artificial intelligence to understand human language in orthopedics

“…Within academia, advanced NLP-based applications are typically developed to perform an array of tasks including information retrieval, information extraction, and cohort identiication [23]. Orthopedic researchers have recently found success in using these techniques for variable extraction, classiication, and cohort identiication tasks [4,21,42,44,48,50,55]. Given the growing inluence of NLP on everyday life, both personal and professional, musculoskeletal health professionals need to possess a foundational understanding of how these techniques function, and how they may be applied to their work.…”

A Natural Language Processing Pipeline to Study Disparities in Cannabis Use and Documentation Among Children and Young Adults A Survey of 21 Years of Electronic Health Records

“…Hence, many of the state-of-the-art NLP models do not perform as well when applied directly to clinical notes and the data needs to be thoroughly preprocessed and cleaned before being fed into the models. Studies such as Tavabi et al [16], Wang et al [17], Ling et al [18], developed and evaluated such NLP pipelines and approaches on clinical notes for purposes like cohort identification and building registries. Some studies have also used NLP to identify substance use from clinical notes.…”