Use of unstructured text in prognostic clinical prediction models: a systematic review

Abstract

Objective This systematic review aims to assess how information from unstructured text is used to develop and validate clinical prognostic prediction models. We summarize the prediction problems and methodological landscape and determine whether using text data in addition to more commonly used structured data improves the prediction performance. Materials and Methods We searched Embase, MEDLINE, Web of Science, and Google Scholar to identify studies that developed prognostic prediction models using information extracted from unstructured text in a data-driven manner, published in the period from January 2005 to March 2021. Data items were extracted, analyzed, and a meta-analysis of the model performance was carried out to assess the added value of text to structured-data models. Results We identified 126 studies that described 145 clinical prediction problems. Combining text and structured data improved model performance, compared with using only text or only structured data. In these studies, a wide variety of dense and sparse numeric text representations were combined with both deep learning and more traditional machine learning methods. External validation, public availability, and attention for the explainability of the developed models were limited. Conclusion The use of unstructured text in the development of prognostic prediction models has been found beneficial in addition to structured data in most studies. The text data are source of valuable information for prediction model development and should not be neglected. We suggest a future focus on explainability and external validation of the developed models, promoting robust and trustworthy prediction models in clinical practice.

Multimodal deep learning for COVID-19 prognosis prediction in the emergency department: a bi-centric study.

“…The current results suggest that NLP may also be used for prognosis prediction purposes in other widespread diseases in the Emergency setting, in keeping with a recent systematic review 22 .…”

“…To the best of our knowledge, using textual data to predict the outcome of patients with SARS-CoV-2 has not yet been fully explored. Presently, the methodology employed for analyzing textual data, i.e., NLP, has been mostly used for entity recognition, literature-based discovery, and question answering 21,22 . The importance of including textual data in prognosis prediction was previously addressed by Silverman et al, 23 .…”

Predicting future falls in older people using natural language processing of general practitioners’ clinical notes

“…However, contrary to the findings of [ 29 ], the performance of our prediction model based only on the unstructured clinical notes did not exceed that of the model with just the clinical variables. A possible explanation is the differences in clinical documentation of both the structured and unstructured data across clinical settings [ 32 ]. The unstructured clinical notes in a particular intensive care setting usually contain different types of notes (e.g.…”

Prediction of High-Risk Donors for Kidney Discard and Nonrecovery Using Structured Donor Characteristics and Unstructured Donor Narratives

“…Natural language processing (NLP) techniques can extract relevant information from free-text narratives and incorporate it into predictive models. Basic models such as the bag-of-words model have been used in NLP in the medical field, yet accurate predictions require a large amount of training data. More recently, however, transformer-based models have emerged, eg, Bidirectional Encoder Representations From Transformers (BERT [Google]) and Generative Pretrained Transformer (GPT [OpenAI]) .…”