A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies

Abstract

Background: Multicentric approaches are widely used in clinical trials to assess generalizability of findings, however they are novel in laboratory-based experimentation. It is unclear how multilaboratory studies may differ in conduct and results from single lab studies. Here we synthesized characteristics of these studies and quantitatively compared their outcomes to those generated by single laboratory studies.Methods: MEDLINE and Embase were systematically searched. Screening and data extractions were completed in duplicate by independent reviewers. Multilaboratory studies investigating interventions using in vivo animal models were included. Study characteristics were extracted. Systematic searches were then performed to identify single center studies matched by intervention and disease. Difference in standardized mean differences (DSMD) was then calculated across studies to assess differences in effect estimates based on study design (>0 indicates larger effects in single center studies).Results: Sixteen multilaboratory studies met inclusion criteria and were matched to 100 single center studies. The multicenter study design was applied across a diverse range of diseases, including traumatic brain injury, myocardial infarction, and diabetes. The median number of centers was 4 (range 2-6) and the median sample size was 111 (range 23-384) with rodents most frequently used. Multicenter studies adhered to practices that reduce risk of bias significantly more often than single center studies. Multicenter studies also demonstrated significantly smaller effect sizes than single center studies (DSMD 0.72 [95% confidence interval 0.43-1]).Conclusion: Multilaboratory studies demonstrate trends that have been well recognized in clinical research (i.e. smaller treatment effects with multicentric evaluation and greater rigour in study design). This approach may provide a method to robustly assess interventions and generalizability of findings between laboratories.Funding: uOttawa Junior Clinical Research Chair; The Ottawa Hospital Anesthesia Alternate Funds Association; Canadian Anesthesia Research Foundation; Government of Ontario Queen Elizabeth II Graduate Scholarship in Science and Technology.Clinical trial registration: PROSPERO CRD4201809398.

A Review of Multisite Replication Projects in Social Psychology: Is It Viable to Sustain Any Confidence in Social Psychology’s Knowledge Base?

“…It is highly likely that effect sizes reported throughout the social-psychology literature are frequently inflated because of publication bias and p -hacking. The shrinking of effect sizes has also been noted in medical research (Hunniford et al, in press), where it was attributed to superior methodological rigor, including larger samples, in multisite replications than in single-site original studies. Fiedler and Prager (2018) proposed that regression toward the mean (i.e., the statistical tendency for extreme observations or results to be followed by others closer to the mean) almost guarantees that replications (whether multilab or single lab) will produce smaller effects than original studies.…”

Animal research in cardiac arrest

“… 40 , 41 In a systematic review from 2023, 16 multicenter animal studies across all research disciplines were identified. 42 Twelve of 16 studies were published in 2015–2020 illustrating an increased focus in the recent years. Interestingly, the review identified that study quality was higher for multilaboratory studies versus single laboratory studies, although none of the identified multilaboratory had low risk of bias.…”

“…This will reduce heterogeneity but at the cost of external validity. 42 This is also illustrated in the cardiac arrest multilaboratory study where experimental procedures such as anesthesia and chest compression techniques differed between laboratories. Furthermore, multilaboratory studies are costly and whether they improve identification of treatments that results in successful clinical translation is unknown.…”

Enhancing the study on antithrombotic therapy after stroke in AF patients

“…Multi-centre studies can provide a more diverse patient population, reducing selection bias, and enhancing the external validity of the findings. 7 We are interested in understanding the feasibility and potential challenges of expanding this research to include multiple centres.…”