Meta-analysis on reporting practices as a source of heterogeneity in in vitro cancer research

Abstract

ObjectivesHeterogeneity of results of exact same research experiments oppose a significant socioeconomic burden. Insufficient methodological reporting is likely to be one of the contributors to results heterogeneity; however, little knowledge on reporting habits of in vitro cancer research and their effects on results reproducibility is available. Exemplified by a commonly performed in vitro assay, we aim to fill this knowledge gap and to derive recommendations necessary for reproducible, robust and translational preclinical science.MethodsHere, we use systematic review to describe reporting practices in in vitro glioblastoma research using the Uppsala-87 Malignant Glioma (U-87 MG) cell line and perform multilevel random-effects meta-analysis followed by meta-regression to explore sources of heterogeneity within that literature, and any associations between reporting characteristics and reported findings. Literature that includes experiments measuring the effect of temozolomide on the viability of U-87 MG cells is searched on three databases (Embase, PubMed and Web of Science).ResultsIn 137 identified articles, the methodological reporting is incomplete, for example, medium glucose level and cell density are reported in only 21.2% and 16.8% of the articles. After adjustments for different drug concentrations and treatment durations, the results heterogeneity across the studies (I2=68.5%) is concerningly large. Differences in culture medium glucose level are a driver of this heterogeneity. However, infrequent reporting of most experimental parameters limits the analysis of reproducibility moderating parameters.ConclusionsOur results further support the ongoing efforts of establishing consensus reporting practices to elevate durability of results. By doing so, this work can raise awareness of how stricter reporting may help to improve the frequency of successful translation of preclinical results into human application. The authors received no specific funding for this work. A preregistered protocol is available at the Open Science Framework (https://osf.io/9k3dq).

Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier

“…However, challenges in translating findings from in vitro research to the clinic may hinder efforts to replace animal research. Poor reporting of measures to reduce the risk of bias in in vitro studies may contribute to this translational challenge [ 2 ], and research which systematically identifies such issues [ 3 ] may lead to improvements in the design, conduct and reporting of in vitro research, and, thereby, their adoption as alternatives to animal research.…”

“…The application of systematic review methodologies to in vivo animal studies has, similarly, identified opportunities for improvement [ 6 , 7 ]. More recently, reviews of in vitro data have suggested similar problems may be prevalent there [ 2 , 3 , 8 ].…”

Reporting In Vitro Experiments Responsibly – the RIVER Recommendations

“…Evidence suggests that in vitro studies are often not reported with sufficient transparency for readers to appraise their methodological rigour and interpret the results. Two recent systematic reviews of drug discovery studies using cancer cell lines highlight the scale of the issue [3,4]: the majority of papers analysed in these reviews lacked basic information about experimental procedures, such as the type of control used, how cell lines were identified, cell passage numbers or the concentration of key medium constituents. In one analysis, a quarter of the papers also lacked even the most rudimentary information on cell culture conditions, such as cell density, carbon dioxide concentration or temperature.…”

“…Significant differences between the results of the studies analysed could not be explained based on the limited experimental information reported in the papers, which raises concerns about the reliability of these studies and their translational value. Most in vitro studies also lack information pertaining to the internal validity of the experiments, such as details about whether randomisation or blinding were used, the criteria used to exclude data, or how sample sizes were determined [4,5]. Without clarity on the experimental approach and statistical methods used, a reader cannot evaluate the extent to which bias may have influenced a study's results, whether the analysis is appropriate, or how generalisable the findings may be.…”

Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier

“…The application of systematic review methodologies to in vivo animal studies has, similarly, identified opportunities for improvement [5,6]. More recently, reviews of in vitro data have suggested similar problems may be prevalent there [2,7,8].…”