Data extraction error in pharmaceutical versus non-pharmaceutical interventions for evidence synthesis: Study protocol for a crossover trial

Abstract

BackgroundData extraction is the foundation for a trusted systematic review; it is often time- and labor-consuming and prone to errors. Whilst data extraction errors frequently occur in literatures, an interesting phenomenon was observed that such erroneous risks tend to be higher in trials of pharmaceutical interventions compared to non-pharmaceutical one. This phenomenon has not been verified by high-quality evidence; The elucidation of which would have implications for guidelines, practice and policy.Methods and analysesWe prepare a 1:1 designed, randomized, multicenter, investigator-blinded, cross-over trial to elucidate the potential variants on the error rates of meta-analyses with pharmaceutical against non-pharmaceutical interventions. Eligible participants that at the 2nd year of their post-graduate program or above (e.g., doctoral program) will be recruited and randomly assigned to one of the two groups to complete pre-defined data extraction tasks: 1) group A contains 10 randomized controlled trials (RCTs) of pharmaceutical interventions; 2) group B contains 10 RCTs of non-pharmaceutical interventions. Participants who would finish the data extraction will be further assigned to the alternative group for another round of data extraction, after at least 30 mins break. Finally, those participants assigned to A or B group will be further 1:1 randomly matched based on a random-sequenced number, for the double-checking process on the extracted data. The primary outcome will be the error rates on data extraction of the pharmaceutical intervention group and the non-pharmaceutical group, before the double-checking process, in terms of the cell level, study level, and participant level. The secondary outcome will be the error rates on data extraction of the pharmaceutical intervention group and the non-pharmaceutical group, after the double-checking process, again, in terms of the cell level, study level, and participant level. The generalized linear mixed effects model (based on the above three levels) will be used to estimate the potential differences on the error rates, with a link function of log that refers to binomial distribution. Subgroup analyses are planned by the following factors: the experience of individuals on systematic reviews, time used for the data extraction.Ethics and disseminationThis study has been approved by the institutional review board of Anhui Medical University. We plan to present our findings at international scientific meetings, and publish our findings in a high-quality peer-reviewed academic journal.Trial registrationChinese Clinical Trial Register Center (Identifier: ChiCTR2200062206).Strengths and limitations of the studyThis will be the first trial to test the error rates of data extraction in the sight of intervention type that characterized by pharmaceutical intervention or non-pharmaceutical intervention.This will be the third randomized trial on strategy of data extraction in the world and the first in Asian-Pacific area.The use of a cross-over design provides a valid way to eliminate the potential impact of the heterogeneous contexts of the studies and thus is expected to provide less biased evidence to support a better evidence synthesis practice.The restrict the participants to 2nd year post-graduate students or above to ensure the feasibility of the trial, which is no doubt impact the representativeness of the samples.A group of useful strategies should be taken to minimize the impact of the possible sharing of completed extraction table among the participants.

Double data extraction was insufficient for minimizing errors in evidence synthesis: a randomized controlled trial

“…83220405), and has been registered with the Chinese Clinical Trial Registry Center (Identifier: ChiCTR2200062206). The protocol for the trial was published previously [15], and findings are reported following the CONSORT 2010 statement and its extension to randomized crossover trials [16].…”