Is Economics Research Replicable? Sixty Published Papers From Thirteen Journals Say “Often Not”

Chang, Andrew C.; Li, Phillip

doi:10.1561/104.00000053

Cited by 36 publications

(23 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Pursuing a comparative approach is an informal test of this presumed external validity. Put differently, in response to a lack of replication and/or replicability in economics and psychology (Earp and Trafimow, 2015;Chang and Li, 2018), using UKHLS data alongside GSOEP data can be viewed as a replication of what is thought to be known from GSOEP data. To make results comparable I use the same set of controls for the UK as I did for Germany.…”

Section: Results For the Ukmentioning

confidence: 99%

“…Given that it often seems in the literature as though results obtained from surveys that are representative of the population of one particular country are generalizable to all industrialized countries, it is especially important to investigate whether this implicit assumption does in fact hold. This is particularly true today, when it has become apparent that many results in psychology and economics are not replicable (Earp and Trafimow, 2015;Chang and Li, 2018), while there is still a lack of attempts at replication (Hamermesh, 2007).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

People do not adapt. New analyses of the dynamic effects of own and reference income on life satisfaction

Kaiser

2020

Journal of Economic Behavior & Organization

View full text Add to dashboard Cite

Do people adapt to changes in income? In contradiction to much of the previous literature, I find no evidence of adaptation to income in GSOEP (1984-2015) and UKHLS (1996-2017) data. Furthermore, I find that people also do not adapt to changes in reference income. Instead, reference income effects may be subject to reinforcement over time. Following the empirical approach of Vendrik (2013), I obtain these findings by estimating life satisfaction equations in which contemporaneous and lagged terms for a respondent's own household income and their estimated reference income are simultaneously entered. Additionally, I instrument for own income and include lags of a large set of controls. What was found to be adaptation to raw household income in previous studies turns out to have been driven by reinforcement of an initially small negative effect of household size that grows large over time. Implications of this result for the estimation of equivalence scales with subjective data are discussed.

show abstract

Section: Results For the Ukmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

People do not adapt. New analyses of the dynamic effects of own and reference income on life satisfaction

Kaiser

2020

Journal of Economic Behavior & Organization

View full text Add to dashboard Cite

show abstract

“…We need to be able to assess the validity of past research to push forward with new. In fact, attempts at replication often fail to confirm widely accepted findings (Chang & Li, 2018;Duvendack, Palmer-Jones, & Reed, 2017), to the extent that when Aarts et al (2015) looked at the replication of 100 studies published in 2008 in highly ranked psychology journals, they uncovered a confirmation rate of only about 39%. Similarly, Camerer et al (2018) find a significant effect in the same direction of the original study for 62% of the studies, and the effect size of the replications is on average about 50% of the original effect size.…”

Section: Enabling Cumulative Knowledge and Learningmentioning

confidence: 99%

A new approach to data access and research transparency (DART)

2020

View full text Add to dashboard Cite

Recent debates on transparency and replicability suggest that JIBS needs to update its approach on data access and research transparency (DART). We propose a series of initiatives, knowing well that there is a balance to be struck. There are clear benefits on the one hand, chief among these the potential for learning and knowledge accumulation, and equally manifest challenges on the other: the imperative to respect privacy, confidentiality, and intellectual property rights. Without addressing these challenges, will there be the highquality data on which the benefits depend? We present access and transparency objectives, and set out how an actionable and effective approach towards DART will be implemented, but also address ethical, legal, and organizational challenges of concern to us as a scholarly community. Journal of International Business Studies (2020).

show abstract

“…It is now known that across scientific disciplines, including biology (Andrew et al, 2015), epidemiology (Begley and Ioannidis, 2015), genetics (Ioannidis et al, 2009), and economics (Camerer et al, 2016;Christensen and Miguel, 2018;Chang and Li, 2015), scientific results often do not reproduce or replicate. In the behavioural sciences in particular, the "replication crisis" of psychology (Pashler and Wagenmakers, 2012) has spurred much research on the causes and consequences of non-replication and irreproducibility (OSF, 2015;Maxwell et al, 2015;Klein et al, 2018;Muthukrishna and Henrich, 2019;Smaldino and McElreath, 2016;Nissen et al, 2016;Higginson and Munafa, 2016).…”

Section: Introductionmentioning

confidence: 99%

Reproducibility improves exponentially over 63 years of social learning research

Minocher

Atmaca

Bavero

et al. 2020

Preprint

View full text Add to dashboard Cite

Interest in improving reproducibility, replicability and transparency of research has increased substantially across scientific fields over the last few decades. We surveyed 560 empirical, quantitative publications published between 1955 and 2018, to estimate the rate of reproducibility for research on social learning, a large subfield of behavioural ecology. We found supporting materials were available for less than 30% of publications during this period. The availability of data declines exponentially with time since publication, with a half-life of about six years, and this “data decay rate” varies systematically with both study design and study species. Conditional on materials being available, we estimate that a reasonable researcher could expect to successfully reproduce about 80% of published results, based on our evaluating a subset of 40 publications. Taken together, this indicates an overall success rate of 24% for both acquiring materials and recovering published results, with non- reproducibility of results primarily due to unavailable, incomplete, or poorly documented data. We provide recommendations to improve the reproducibility of research on the ecology and evolution of social behaviour.

show abstract

Is Economics Research Replicable? Sixty Published Papers From Thirteen Journals Say “Often Not”

Cited by 36 publications

References 47 publications

People do not adapt. New analyses of the dynamic effects of own and reference income on life satisfaction

People do not adapt. New analyses of the dynamic effects of own and reference income on life satisfaction

A new approach to data access and research transparency (DART)

Reproducibility improves exponentially over 63 years of social learning research

Contact Info

Product

Resources

About