Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them

Garattini, Silvio; Jakobsen, Janus Christian; Wetterslev, Jørn; Bertelè, Vittorio; Banzi, Rita; Rath, Ana; Neugebauer, Edmund; Laville, Martine; Masson, Yvonne; Hivert, Virginie; Eikermann, Michaela; Aydın, Barbaros; Ngwabyt, Sandra; Martinho, Clarisse; Gerardi, Chiara; Szmigielski, Cezary; Demotes-Mainard, Jacques; Gluud, Christian

doi:10.1016/j.ejim.2016.03.020

Cited by 96 publications

(108 citation statements)

References 94 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…We used central randomisation stratifying for important predictive factors,51 52 53 54 used blinded outcome assessors,51 52 53 54 assessed several register based outcomes that are likely less influenced by knowledge about intervention group affiliation than other outcomes, included stratification factors in our main analyses,55 56 conducted our main analyses on the data where missingness was controlled by multiple imputation,57 58 conducted our analyses blinded for intervention group,59 and drew our conclusions without knowledge of intervention group 59. Moreover, although we based our sample size calculation on a power of 80%, by inflating our sample by 30% and analysing all participants using multiple imputations, we actually had a power of 91% to detect the a priori defined least significant difference regarding the primary outcome.…”

Section: Discussionmentioning

confidence: 99%

“…From top to bottom, graphs show development of negative symptoms (mean score on the scale for assessment of negative symptoms (sans)), psychotic symptoms (mean score on the scale for assessment for positive symptoms (saPs)), functional level (mean score on the personal and social performance scale (PsP)), and cognitive functioning (mean total score on the brief assessment of cognition in schizophrenia (baCs)) from baseline to follow-up strengths and limitations of study Strengths We used central randomisation stratifying for important predictive factors, 51-54 used blinded outcome assessors, [51][52][53][54] assessed several register based outcomes that are likely less influenced by knowledge about intervention group affiliation than other outcomes, included stratification factors in our main analyses, 55 56 conducted our main analyses on the data where missingness was controlled by multiple imputation, 57 58 conducted our analyses blinded for intervention group, 59 and drew our conclusions without knowledge of intervention group. 59 Moreover, although we based our sample size calculation on a power of 80%, by inflating our sample by 30% and analysing all participants using multiple imputations, we actually had a power of 91% to detect the a priori defined least significant difference regarding the primary outcome.…”

Section: Fig 2 |mentioning

confidence: 99%

See 1 more Smart Citation

Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II)

Albert

Melau

Jensen

et al. 2017

BMJ

Self Cite

View full text Add to dashboard Cite

Objective To compare the effects of five years of specialised early intervention (SEI) treatment for first episode schizophrenia spectrum disorder with the standard two years of SEI plus three years of treatment as usual.Design Randomised, superiority, parallel group trial with blinded outcome assessment. Randomisation was centralised and computerised with concealed randomisation sequence carried out at an external site.Setting Participants were recruited from six OPUS teams in Denmark between 2009 and 2012. OPUS teams provide SEI treatment to all patients diagnosed with a schizophrenia spectrum disorder in Denmark.Participants 400 participants (51% women) with a mean age of 25.6 (standard deviation 4.3) were randomised to five years of SEI (experimental intervention; n=197) or to two years of SEI plus three years of treatment as usual (control; n=203).Interventions OPUS treatment consists of three core elements—modified assertive community treatment, family involvement, and social skill training—with a patient-case manager ratio of no more than 12:1. For participants randomised to five years of OPUS treatment, the treatment was largely unchanged. Participants randomised to the control group were mostly referred to community health centres after two years of SEI treatment.Main outcomes Follow-up assessments were conducted five years after start of OPUS treatment. Primary outcome was negative symptoms measured on the scale for assessment of negative symptoms (avolition-apathy, anhedonia, alogia, and affective blunting). Secondary outcomes were remission of both negative and psychotic symptoms, psychotic symptoms, suicidal ideation, substance abuse, compliance with medical treatment, adherence with treatment, client satisfaction, days in hospital care, and labour market affiliation.Results Levels of negative symptoms did not differ between the intervention group and control group (1.72 v 1.81 points; estimated mean difference −0.10 (95% confidence interval 0.33 to 0.13), P=0.39). Participants receiving five years of OPUS treatment were more likely to remain in contact with specialised mental health services (90.4% v 55.6%, P<0.001), had higher client satisfaction (estimated mean difference 2.57 points (95% confidence interval 1.36 to 3.79), P<0.001), and had a stronger working alliance (estimated mean difference 5.56 points (95% confidence interval 2.30 to 8.82), P=0.001) than the control group.Conclusions This trial tests SEI treatment for up to five years for patients with first episode schizophrenia spectrum disorder; previous trials have found treatment effects for programmes lasting from one to three years. The prolonged SEI treatment had few effects, which could be due to the high level of treatment provided to control participants and the late start of specialised treatment.Trial registration Clinicaltrial.gov NCT00914238.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Fig 2 |mentioning

confidence: 99%

Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II)

Albert

Melau

Jensen

et al. 2017

BMJ

Self Cite

View full text Add to dashboard Cite

show abstract

“…This point of view seems to unduly emphasise the difference between the consequences of an interim-analysis in a single trial and the consequences of a sequential meta-analysis of several trials. First, the systematic review is placed at the top of the generally recognised hierarchy of evidence, meaning that the systematic review is considered the most likely reliable source of evidence, implicating whether an intervention should be implemented in clinical practice or further trials should be launched [52, 53]. Interventions are often recommended in clinical guidelines and implemented in clinical practice when a meta-analysis shows statistical significance on the traditional naïve level ( P < 0.05) [16, 18, 67–69].…”

Section: Discussionmentioning

confidence: 99%

Trial Sequential Analysis in systematic reviews with meta-analysis

Wetterslev

Jakobsen

Gluud

2017

BMC Med Res Methodol

Self Cite

850

558

View full text Add to dashboard Cite

BackgroundMost meta-analyses in systematic reviews, including Cochrane ones, do not have sufficient statistical power to detect or refute even large intervention effects. This is why a meta-analysis ought to be regarded as an interim analysis on its way towards a required information size. The results of the meta-analyses should relate the total number of randomised participants to the estimated required meta-analytic information size accounting for statistical diversity. When the number of participants and the corresponding number of trials in a meta-analysis are insufficient, the use of the traditional 95% confidence interval or the 5% statistical significance threshold will lead to too many false positive conclusions (type I errors) and too many false negative conclusions (type II errors).MethodsWe developed a methodology for interpreting meta-analysis results, using generally accepted, valid evidence on how to adjust thresholds for significance in randomised clinical trials when the required sample size has not been reached.ResultsThe Lan-DeMets trial sequential monitoring boundaries in Trial Sequential Analysis offer adjusted confidence intervals and restricted thresholds for statistical significance when the diversity-adjusted required information size and the corresponding number of required trials for the meta-analysis have not been reached. Trial Sequential Analysis provides a frequentistic approach to control both type I and type II errors. We define the required information size and the corresponding number of required trials in a meta-analysis and the diversity (D2) measure of heterogeneity. We explain the reasons for using Trial Sequential Analysis of meta-analysis when the actual information size fails to reach the required information size. We present examples drawn from traditional meta-analyses using unadjusted naïve 95% confidence intervals and 5% thresholds for statistical significance. Spurious conclusions in systematic reviews with traditional meta-analyses can be reduced using Trial Sequential Analysis. Several empirical studies have demonstrated that the Trial Sequential Analysis provides better control of type I errors and of type II errors than the traditional naïve meta-analysis.ConclusionsTrial Sequential Analysis represents analysis of meta-analytic data, with transparent assumptions, and better control of type I and type II errors than the traditional meta-analysis using naïve unadjusted confidence intervals.

show abstract

“…Another potential limitation when using composite outcomes, such as serious adverse events, is that each component of a composite outcome will not have similar degrees of severity, so event severity differences between the compared groups might not be shown in the analysis of such outcomes 18. Finally, we found no evidence of a difference when assessing all-cause mortality, cardiovascular mortality and myocardial infarction, and Trial Sequential Analyses showed that we do not have enough information to confirm or reject our anticipated intervention effects.…”

Section: Limitationsmentioning

confidence: 99%

Cochrane Corner: drug-eluting stents versus bare-metal stents for acute coronary syndrome

Feinberg

Gluud

Jakobsen

2018

Heart

Self Cite

View full text Add to dashboard Cite

Evidence-based clinical practice: Overview of threats to the validity of evidence and how to minimise them

Cited by 96 publications

References 94 publications

Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II)

Five years of specialised early intervention versus two years of specialised early intervention followed by three years of standard treatment for patients with a first episode psychosis: randomised, superiority, parallel group trial in Denmark (OPUS II)

Trial Sequential Analysis in systematic reviews with meta-analysis

Cochrane Corner: drug-eluting stents versus bare-metal stents for acute coronary syndrome

Contact Info

Product

Resources

About