Controversy and Debate: Questionable utility of the relative risk in clinical research: Paper 1: A call for change to practice

Abstract

Background and Objectives: In clinical trials, the relative risk or risk ratio (RR) is a mainstay of reporting of the effect magnitude for an intervention. The RR is the ratio of the probability of an outcome in an intervention group to its probability in a control group. Thus, the RR provides a measure of change in the likelihood of an event linked to a given intervention. This measure has been widely used because it is today considered a measure with ''portability'' across varying outcome prevalence, especially when the outcome is rare. It turns out, however, that there is a much more important problem with this ratio, and this paper aims to demonstrate this problem.Methods: We used mathematical derivation to determine if the RR is a measure of effect magnitude alone (i.e., a larger absolute value always indicating a stronger effect) or not. We also used the same derivation to determine its relationship to the prevalence of an outcome. We confirm the derivation results with a follow-up analysis of 140,620 trials scraped from the Cochrane.Results: We demonstrate that the RR varies for reasons other than the magnitude of the effect because it is a ratio of two posterior probabilities, both of which are dependent on baseline prevalence of an outcome. In addition, we demonstrate that the RR shifts toward its null value with increasing outcome prevalence. The shift toward the null happens regardless of the strength of the association between intervention and outcome. The odds ratio (OR), the other commonly used ratio, measures solely the effect magnitude and has no relationship to the prevalence of an outcome in a study nor does it overestimate the RR as is commonly thought.Conclusions: The results demonstrate the need to (1) end the primary use of the RR in clinical trials and meta-analyses as its direct interpretation is not meaningful, (2) replace the RR by the OR, and (3) only use the postintervention risk recalculated from the OR for any expected level of baseline risk in absolute terms for purposes of interpretation such as the number needed to treat. These results will have far-reaching implications such as reducing misleading results from clinical trials and meta-analyses and ushering in a new era in the reporting of such trials or meta-analyses in practice.

Prevalence and impact of cardiac injury on COVID‐19: A systematic review and meta‐analysis

“…The effect measurement estimate of mortality risk chosen was the odds ratio (ORs) in our study. Risk ratio and hazard ratio in other studies were considered to be OR, as OR was shown to be a more effective measure 28 . If not available, the ORs were calculated by events and total numbers of patients in two groups.…”

Making Patient-Specific Treatment Decisions Using Prognostic Variables and Utilities of Clinical Outcomes

“…An intense debate is ongoing, with some considering the non-collapsibility of ORs and HRs to be an advantage for statistical modeling because the mathematical features that render a parameter non-collapsible also provide the properties that facilitate statistical analyses on the estimation scale, including the unlimited range of odds and hazard rates and the fact that they are non-linear functions of probabilities [ 9 , 22 ]. Others, however, argue that collapsible measures such as risk difference and mean survival time difference are preferable parameters to use because, in addition to being difficult to interpret clinically, the non-collapsibility of ORs and HRs adds to problems of transportability and makes it harder to compare estimates across studies [ 17 , 23 ].…”

Effect Modifiers and Statistical Tests for Interaction in Randomized Trials

“…For effect measures on a multiplicative scale (such as risk ratio, hazard ratio, or odds ratio) as opposed to the additive scale (such as risk differences), the analyses should be performed using the log-transformation and with the corresponding standard errors [3]. Importantly, effect modification may be present on one scale but not on another, and conflicting opinions exist on which scale to use [6]. The European Medicines Agency (EMA) recommends using the scale on which the endpoint is commonly analyzed, and to present supplementary analyses on the complementary scale where inconsistency is observed [7].…”