EA3: A softmax algorithm for evidence appraisal aggregation

Abstract

Real World Evidence (RWE) and its uses are playing a growing role in medical research and inference. Prominently, the 21st Century Cures Act—approved in 2016 by the US Congress—permits the introduction of RWE for the purpose of risk-benefit assessments of medical interventions. However, appraising the quality of RWE and determining its inferential strength are, more often than not, thorny problems, because evidence production methodologies may suffer from multiple imperfections. The problem arises to aggregate multiple appraised imperfections and perform inference with RWE. In this article, we thus develop an evidence appraisal aggregation algorithm called EA3. Our algorithm employs the softmax function—a generalisation of the logistic function to multiple dimensions—which is popular in several fields: statistics, mathematical physics and artificial intelligence. We prove that EA3 has a number of desirable properties for appraising RWE and we show how the aggregated evidence appraisals computed by EA3 can support causal inferences based on RWE within a Bayesian decision making framework. We also discuss features and limitations of our approach and how to overcome some shortcomings. We conclude with a look ahead at the use of RWE.

E‐synthesis for carcinogenicity assessments: A case study of processed meat

“…The formulae for conditional probabilities of observing a positive effect size given that the indicator holds, respectively, fails to hold, thus are $$$P({ES}=1| {Ind})=0.5+\frac{A+{SS}}{4}P({ES}=1| \bar{{Ind}})=0.5-\frac{A+{SS}}{4}.$$$ These simplistic formulae exemplify the working of E‐Synthesis. More work is required to obtain formulae more suitable for real‐world assessments 39 …”

Fast Methods for Drug Approval: Research Perspectives for Pandemic Preparedness

“…It is built upon the fact that accumulating evidence overrides prior beliefs and the posterior beliefs based on truth-conducive evidence converging to the truth [111]. This puts one in a position to conduct evidence synthesis based on all the available evidence [112][113][114][115] by calculating the probabilities of effectiveness and safety of drugs. Unfortunately, Bayesian evidence synthesis requires the specification of a large number of relevant probabilities, which cannot be defined in an objectively correct way and must be fixed subjectively [116].…”