Generalizing experimental results by leveraging knowledge of mechanisms

Cinelli, Carlos; Pearl, Judea

doi:10.1007/s10654-020-00687-4

Cited by 18 publications

(27 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Methods for transporting evidence of the effectiveness of compound treatments to clinical practice have been proposed ( 26 ). Transportability of evidence may also depend on differences in the mechanisms that determine the outcome in the study and the target populations ( 27 ).…”

Section: Resultsmentioning

confidence: 99%

Applicability of evidence from randomized controlled trials and systematic reviews to clinical practice: A conceptual review

Malmivaara¹

2021

J Rehabil Med

View full text Add to dashboard Cite

Background The value of randomized controlled trials is dependent on the applicability of their findings to clinical decision-making. The aim of this study is to determine a definition and principles for the applicability of evidence from randomized controlled trials and systematic reviews. Methods This narrative review searched studies from PubMed and Web of Science databases using Cochrane Collaboration’s Qualitative Evidence Syntheses guidance. Empirical studies were excluded. Based on the included studies, a definition for the concept and propositions for principles of applicability were formulated. Results A definition and 11 propositions are presented, 6 propositions having additional sub-propositions. Low risk of bias, ability to answer to specific questions, documentation of the details of how randomized controlled trials turned out, reporting of favourable and adverse outcomes, and systematic comparison of randomized controlled trials and clinical data were considered important. Biomedical randomized controlled trials have the widest applicability, while heterogeneity in study characteristics, human perception, behaviour, environmental, equity factors, and health economic issues lessen applicability. Obtaining applicable evidence is a gradual process. Methodological and substance expertise is necessary for assessing applicability. Discussion A definition of applicability and requirements for applicable evidence from randomized controlled trials to real-world contexts are presented. Propositions are suggested for any assessment of applicability of findings from randomized controlled trials, systematic reviews and meta-analyses. LAY ABSTRACT Clinicians’ need for knowledge about a specific patient (or group of patients) is the underlying principle for applicability. Consequently, randomized controlled trials and systematic reviews should document all essential factors needed for clinical decision-making. Documentation of the study protocol (inclusion and exclusion criteria of patients, description of content of interventions and the outcome measures) is not sufficient. The documentation must also cover what actually happened in the randomized controlled trial, i.e. the characteristics of patients, the adherence to the index and control interventions, and the amount of co-interventions. Clinical registers using uniform documentation with randomized controlled trials increase the applicability of the research findings to clinical practice. The broadest applicability of findings comes from randomized controlled trials that assess the effectiveness of a single biological intervention for a well-defined disease using a valid biological outcome measure. Heterogeneity in study characteristics (patients, interventions and outcomes), and the presence of human perception (diagnosis, interventions and outcomes based on patient perception), and behaviour, environmental and equity fact...

show abstract

Section: Resultsmentioning

confidence: 99%

Applicability of evidence from randomized controlled trials and systematic reviews to clinical practice: A conceptual review

Malmivaara¹

2021

J Rehabil Med

View full text Add to dashboard Cite

show abstract

“…Op deze manier dragen ook de besproken onderzoeken bij aan de kennis over het mechanisme rondom de werking van Megan's Law. Het identificeren van het onderliggende mechanisme versterkt bovendien de kennis over de generaliseerbaarheid van effecten (Cinelli & Pearl, 2020) en de benodigde ondersteunende factoren (Cartwright & Stegenga, 2011). Wat die ondersteunende factoren ofwel randvoorwaarden precies inhouden, wordt verder geïllustreerd in het volgende voorbeeld.…”

Section: Verklarend Mechanismeunclassified

Hoe weet je wat werkt? Over de zoektocht naar bruikbare kennis voor beleidseffectiviteit

Eersel¹

2021

Beleidsonderzoek Online

View full text Add to dashboard Cite

Utrecht. Ze was daarvoor onderzoeker bij de lokale overheid en heeft een achtergrond in de wetenschapsfilosofie. 1Zie www.volkskrant.nl/economie/nobelprijswinnaar-esther-duflo-intuitie-kan-zo-schadelijk-zijn~b4284201/. 2 Zie www.rijksbegroting.nl/beleidsevaluaties/evaluaties-en-beleidsdoorlichtingen/handreikingbeleidsdoorlichtingen/het-meten-van-doeltreffendheid.Dit artikel uit Beleidsonderzoek Online is gepubliceerd door Boom bestuurskunde en is bestemd voor anonieme bezoeker

show abstract

“…Indeed, generalizing -or transporting-causal effects measured on the RCT to a new population is necessary to estimate the target population causal effect if there are effect modifiers with a different distribution in the target population than that in the trial. Using covariates present in both RCT and an observational sample of the target population, this target population average treatment effect (ATE) can be estimated with a variety of methods (Cole and Stuart, 2010;Stuart et al, 2011;Bareinboim and Pearl, 2013;Tipton, 2013;Kern et al, 2016;Bareinboim and Pearl, 2016;Kallus et al, 2018;Dong et al, 2020;Cinelli and Pearl, 2020), reviewed in Colnet et al (2020); Degtiar and Rose (2021). The RCT yields an unbiased estimate of the treatment effect, but subject to a population shift while the observational data at hand gives a unbiased sample of the target population of interest.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore we complete Subsection 2.2 with explicit consistency results that hold under certain assumptions (see in appendix Lemma 8 for the g-formula and Lemma 9 for IPSW along with their proofs in Section B.1.2 and Section B.2). Note the specific usage of a square for S used to depict a special variable creating differences in covariates' distributions between datasets (also called the source) (Pearl and Bareinboim, 2014;Bareinboim and Pearl, 2016;Cinelli and Pearl, 2020).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Causal effect on a target population: a sensitivity analysis to handle missing covariates

Colnet¹,

Josse²,

Scornet³

et al. 2021

Preprint

View full text Add to dashboard Cite

While a randomized controlled trial (RCT) readily measures the average treatment effect (ATE), this measure may need to be shifted to generalize to a different population. Standard estimators of the target population treatment effect are based on the distributional shift in covariates, using inverse propensity sampling weighting (IPSW) or modeling response with the g-formula. However, these need covariates that are available both in the RCT and in an observational sample, which often qualifies very few of them. Here we analyze how the classic estimators behave when covariates are missing in at least one of the two datasets-RCT or observational. In line with general identifiability conditions, these estimators are consistent when including only treatment effect modifiers that are shifted in the target population. We compute the expected bias induced by a missing covariate, assuming Gaussian covariates and a linear model for the conditional ATE function. This enables sensitivity analysis for each missing covariate pattern. In addition, this method is particularly useful as it gives the sign of the expected bias. We also show that there is no gain imputing a partially-unobserved covariate. Finally we study the replacement of a missing covariate by a proxy, and the impact of imputation. We illustrate all these results on simulations, as well as semi-synthetic benchmarks using data from the Tennessee Student/Teacher Achievement Ratio (STAR), and with a real-world example from the critical care medical domain.

show abstract

Generalizing experimental results by leveraging knowledge of mechanisms

Cited by 18 publications

References 19 publications

Applicability of evidence from randomized controlled trials and systematic reviews to clinical practice: A conceptual review

Applicability of evidence from randomized controlled trials and systematic reviews to clinical practice: A conceptual review

Hoe weet je wat werkt? Over de zoektocht naar bruikbare kennis voor beleidseffectiviteit

Causal effect on a target population: a sensitivity analysis to handle missing covariates

Contact Info

Product

Resources

About