Enhancing evidence-based medicine with natural language argumentative analysis of clinical trials

Abstract

In the latest years, the healthcare domain has seen an increasing interest in the definition of intelligent systems to support clinicians in their everyday tasks and activities. Among others, also the field of Evidence-Based Medicine is impacted by this twist, with the aim to combine the reasoning frameworks proposed thus far in the field with mining algorithms to extract structured information from clinical trials, clinical guidelines, and Electronic Health Records. In this paper, we go beyond the state of the art by proposing a new end-to-end pipeline to address argumentative outcome analysis on clinical trials. More precisely, our pipeline is composed of (i) an Argument Mining module to extract and classify argumentative components (i.e., evidence and claims of the trial) and their relations (i.e., support, attack), and (ii) an outcome analysis module to identify and classify the effects (i.e., improved, increased, decreased, no difference, no occurrence) of an intervention on the outcome of the trial, based on PICO elements. We annotated a dataset composed of more than 500 abstracts of Randomized Controlled Trials (RCT) from the MEDLINE database, leading to a labeled dataset with 4198 argument components, 2601 argument relations, and 3351 outcomes on five different diseases (i.e., neoplasm, glaucoma, hepatitis, diabetes, hypertension). We experiment with deep bidirectional transformers in combination with different neural architectures (i.e., LSTM, GRU and CRF) and obtain a macro F1-score of .87 for component detection and .68 for relation prediction, outperforming current state-of-the-art end-to-end Argument Mining systems, and a macro F1-score of .80 for outcome classification.

Using semantics to scale up evidence-based chemical risk-assessments

“…Understanding how a stressor impacts the cell cycle is just one of the many outcomes that a decision maker would consider when establishing public policy around potential carcinogens but cellular level outcomes are just one of the many streams of evidence that includes amongst other endpoints genetic markers, and evidence on humans and animals is weighted treated differently when determining if there is a sufficient amount of evidence to change policy. We also do not attempt to differentiate between major and minor claims, however human inter-rater reliability to establish this distinction has been reported as low [ 26 ].…”

“…The post error analysis found that the text needed to address a query appeared at the end of the abstract, which suggests that sentences were likely from the result and conclusion sections. In contrast to argumentation systems that strive to identify major claims [ 25 ] or to differentiate between major and minor claims, which has been shown to have low inter-rater reliability [ 26 ] we make no judgments regarding the veracity of a claim. Instead, directionality and negation of each claim are show to the decision maker as six discrete steps from refuting to negated refute, neutral, negated neutral, negated support, and finally to supporting evidence.…”

“…Annotators who constructed the GENIA corpus could mark the non-consecutive text, but most argumentation systems do not deal with this phenomenon. For example, in [ 26 ], liver and cardiac toxicities was marked as 1 entity which resulted in low inter-annotator agreement because some entities in lists such as eanthralogia/myalgia were separated, but CCNPs were not. In [ 25 ], the coordination issue was avoided by asking annotator to identify the entire sentence that supported or contradicted a query.…”

GraphNLI: A Graph-based Natural Language Inference Model for Polarity Prediction in Online Debates

“…[9]), identifying the claims, their reasons and relationships between claims from clinical trials to inform medical decision making (e.g. [34]), or detecting fallacies from the transcripts of the United States Presidential Debates [39]. Online debates, such as those on Reddit or Twitter for example, can be converted into the argumentation framework by first defining nodes or arguments corresponding to each well-defined logical claim, and drawing signed edges representing supports or attacks between them.…”

ACTA 2.0: A Modular Architecture for Multi-Layer Argumentative Analysis of Clinical Trials

“…First, it exploits novel state-of-the-art neural models for the analysis of clinical trials, going beyond the link prediction task by labelling argument relations (i.e., to label as attack or support the relations holding between the identified argument components). Second, it goes beyond PICO element identification by allowing for the automatic analysis of the effect of an intervention on the observed outcome parameters [Mayer et al, 2021]. Finally, a refactoring of the ACTA architecture allows now for a modular solution, such that, through a REST API, the different ACTA 2.0 modules are made available to be integrated by external systems.…”