No abstract

Social scientists generally enjoy substantial latitude in selecting measures and models for hypothesis testing. Coupled with publication and related biases, this latitude raises the concern that researchers may intentionally or unintentionally select models that yield positive findings, leading to an unreliable body of published research. To combat this "fishing" problem in medical studies, leading journals now require preregistration of designs that emphasize the prior identification of dependent and independent variables. However, we demonstrate here that even with this level of advanced specification, the scope forfishing is considerable when there is latitude over selection of covariates, subgroups, and other elements of an analysis plan. These concerns could be addressed through the use of a form of comprehensive registration. We experiment with such an approach in the context of an ongoing field experiment for which we drafted a complete “mock report" of findings using fake data on treatment assignment. We describe the advantages and disadvantages of this form of registration and propose that a comprehensive but non-binding approach be adopted as a first step to combat fishing by social scientists. Likely effects of comprehensive but non-binding registration are discussed, the principle advantage being communication rather than commitment, in particular that it generates a clear distinction between exploratory analyses and genuine tests.

The Dodd Frank Act was passed by the US Congress in July 2010 and included a provision—Section 1502—that aimed to break the link between conflict and minerals in the Eastern Democratic Republic of Congo. To date there is only one rigorous quantitative analysis that investigates the impact of Dodd-Frank on local conflict events. Looking at the short-term impact (2011–2012), it finds that the policy backfired. This study builds on a larger, more representative, dataset of mining sites and extends the time horizon by three years (2013–2015). The results indicate that the policy also backfired in the longer run, especially in areas home to gold mines. For territories with the average number of gold mines, the introduction of Dodd-Frank increased the incidence of battles with 44%; looting with 51% and violence against civilians with 28%, compared to pre-Dodd Frank averages. Delving deeper into the impact of the conflict minerals legislation is important, as President Trump suspended the legislation in February 2017 for a two-year period, ordering his administration to replace it with another policy.

Social scientists generally enjoy substantial latitude in selecting measures and models for hypothesis testing. Coupled with publication and related biases, this latitude raises the concern that researchers may intentionally or unintentionally select models that yield positive findings, leading to an unreliable body of published research. To combat this “fishing” problem in medical studies, leading journals now require preregistration of designs that emphasize the prior identification of dependent and independent variables. However, we demonstrate here that even with this level of advanced specification, the scope for fishing is considerable when there is latitude over selection of covariates, subgroups, and other elements of an analysis plan. These concerns could be addressed through the use of a form of comprehensive registration. We experiment with such an approach in the context of an ongoing field experiment for which we drafted a complete “mock report” of findings using fake data on treatment assignment. We describe the advantages and disadvantages of this form of registration and propose that a comprehensive but nonbinding approach be adopted as a first step to combat fishing by social scientists. Likely effects of comprehensive but nonbinding registration are discussed, the principal advantage being communication rather than commitment, in particular that it generates a clear distinction between exploratory analyses and genuine tests.

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. AbstractDo citizens view state and traditional authorities as substitutes or complements? Past work has been divided on this question. Some scholars point to competition between attitudes toward these entities, suggesting substitution, whereas others highlight positive correlations, suggesting complementarity. Addressing this question, however, is difficult, as it requires assessing the effects of exogenous changes in the latent valuation of one authority on an individual's support for another. We show that this quantity-a type of elasticity-cannot be inferred from correlations between support for the two forms of authority. We employ a structural model to estimate this elasticity of substitution using data from 816 villages in the Democratic Republic of Congo and plausibly exogenous rainfall and conflict shocks. Despite prima facie evidence for substitution logics, our model's outcomes are consistent with complementarity; positive changes