The end of the wormwars?

Abstract

Aiken et al. (2014) usefully correct some errors in Miguel and Kremer (2004). Miguel and Kremer (2004) made two key claims: 1) deworming creates positive epidemiological externalities, thus causing estimates of the impact of deworming based on individual randomization to be biased downwards; and 2) deworming increases school participation. The results in Aiken et al. (2014) are consistent with these findings. In addition to direct impacts of deworming treatment on worm infections, Aiken et al. (2014) find externality effects within schools on untreated pupils, as well as externality effects across schools up to 3 km away. Similarly, with regard to school participation, both Miguel and Kremer (2004) and Aiken et al. (2014) find direct effects of deworming and externality effects within schools on untreated pupils. As Aiken et al. (2014) point out, most of the errors they identify (rounding errors or updates to the data set) lead to only small changes in estimated coefficients. The key change is that Miguel and Kremer (2004) measured externalities among schools located within 3-6 km that were among the 12 closest schools, rather than among all schools within 3-6 km, as reported; Aiken et al. (2014) report results including the full set of schools within 3-6 km. With the updated data, there is no evidence that worm infection externalities extend beyond the 12 closest schools to the full set of schools within 6 km, perhaps unsurprisingly given the local nature of disease transmission. We disagree with Aiken et al.'s (2014) conclusion that "there was no evidence of a between-school indirect effect" or an overall effect of deworming on school participation. We show that this conclusion is based on an approach that adds substantial noise to the estimation, by heavily weighting a non-significant 3-6 km externality estimate. This note addresses these and other points, and comments on the current state of deworming evidence. † Acknowledgements: We thank

Campbell replication confirms little or no effect of community deworming

“…between the Campbell and Cochrane reviews here could hardly be stronger. 7 The additional network and subgroup analyses in the Campbell review fi nd no evidence of spillover eff ects, no synergistic eff ects of co-interventions, and no evidence regarding any moderating impact of worm burden or the learning environment. 5 Welch and colleagues encourage a further analysis using individual patient data analysis of deworming trials, although substantive new insights seem unlikely given the evidence of ineff ectiveness for most outcomes across multiple analyses.…”

“…Welch and colleagues' review is a thorough and substantive replication of the Cochrane review, 6,7 with additional sophisticated analyses and adjustments that take into account criticisms levelled by deworming advocates at the current Cochrane edition that we author. Since the fi rst edition 8 of the Cochrane review in 2000, advocates of deworming have ignored, ridiculed, or attacked the various editions.…”

When and how to replicate systematic reviews

“…A second independent Campbell review confirmed the findings of the Cochrane Review (Welch 2017). In a commentary on these reviews, Tovey and colleagues stated that while Cochrane and Campbell have developed policies and processes to limit duplication of e orts, given the contentious nature of this particular question, replication was deemed necessary and justified (Tovey 2016).…”

When is systematic review replication useful, and when is it wasteful?

“…Replication can also serve to purposefully broaden or narrow the research question in existing reviews, particularly for questions that may address sizable benefits or harms. When a Cochrane review on mass deworming was criticized for not including long-term educational and labour outcomes from non-randomized studies, a Campbell replication review extended the scope of the original review and confirmed its findings ( 4 , 5 ). These examples highlight the potential value of planned and purposeful systematic review replication.…”