The Research Reproducibility Crisis and Economics of Science

Maniadis, Zacharias; Tufano, Fabio

doi:10.1111/ecoj.12526

Cited by 20 publications

(10 citation statements)

References 79 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are in the throes of a "reproducibility crisis" in the sciences, if headlines are to be taken at face value (Maniadis and Tufano 2017). Scholars have expressed concern-in social sciences including psychology and economics, as well as in the natural sciences-that published research findings may not prove to be reproducible, or may not be "robust" (Baker, 2016a;Ioannidis, Stanley, and Doucouliagos, 2017).…”

Section: Reproducibilitymentioning

confidence: 99%

Replication Redux: The Reproducibility Crisis and the Case of Deworming

Ozier¹

2019

View full text Add to dashboard Cite

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

show abstract

Section: Reproducibilitymentioning

confidence: 99%

Replication Redux: The Reproducibility Crisis and the Case of Deworming

Ozier¹

2019

View full text Add to dashboard Cite

show abstract

“…Moore 2018) and economics (e.g. Maniadis and Tufano 2017) to management (e.g. Alexander, Miller, and Fielding 2015) and policy (e.g.…”

Section: Introductionmentioning

confidence: 99%

The Open Innovation in Science research field: a collaborative conceptualisation approach

Beck

Bergenholtz

Bogers

et al. 2020

Industry and Innovation

158

View full text Add to dashboard Cite

Openness and collaboration in scientific research are attracting increasing attention from scholars and practitioners alike. However, a common understanding of these phenomena is hindered by disciplinary boundaries and disconnected research streams. We link dispersed knowledge on Open Innovation, Open Science, and related concepts such as Responsible Research and Innovation by proposing a unifying Open Innovation in Science (OIS) Research Framework. This framework captures the antecedents, contingencies, and consequences of open and collaborative practices along the entire process of generating and disseminating scientific insights and translating them into innovation. Moreover, it elucidates individual-, team-, organisation-, field-, and society-level factors shaping OIS practices. To conceptualise the framework, we employed a collaborative approach involving 47 scholars from multiple disciplines, highlighting both tensions and commonalities between existing approaches. The OIS Research Framework thus serves as a basis for future research, informs policy discussions, and provides guidance to scientists and practitioners.

show abstract

“…Data quality is vital for ensuring the accuracy, reliability, and validity of survey findings. Traditional approaches to monitoring data quality have sought to consider both the intrinsic (the data collection tool, implementation and support systems governing its use) and extrinsic factors (weather, enumerator-respondent dynamics, community environment) underpinning survey implementation, and in turn, data quality [ 1 ]. The quality of survey data starts with the selection of the survey institutions that will support the design and development, sampling, and implementation of the instrument.…”

Section: Introductionmentioning

confidence: 99%

Using Machine Learning to Optimize the Quality of Survey Data: Protocol for a Use Case in India

Shah¹,

Mohan²,

Bashingwa³

et al. 2020

JMIR Res Protoc

View full text Add to dashboard Cite

Background Data quality is vital for ensuring the accuracy, reliability, and validity of survey findings. Strategies for ensuring survey data quality have traditionally used quality assurance procedures. Data analytics is an increasingly vital part of survey quality assurance, particularly in light of the increasing use of tablets and other electronic tools, which enable rapid, if not real-time, data access. Routine data analytics are most often concerned with outlier analyses that monitor a series of data quality indicators, including response rates, missing data, and reliability of coefficients for test-retest interviews. Machine learning is emerging as a possible tool for enhancing real-time data monitoring by identifying trends in the data collection, which could compromise quality. Objective This study aimed to describe methods for the quality assessment of a household survey using both traditional methods as well as machine learning analytics. Methods In the Kilkari impact evaluation’s end-line survey amongst postpartum women (n=5095) in Madhya Pradesh, India, we plan to use both traditional and machine learning–based quality assurance procedures to improve the quality of survey data captured on maternal and child health knowledge, care-seeking, and practices. The quality assurance strategy aims to identify biases and other impediments to data quality and includes seven main components: (1) tool development, (2) enumerator recruitment and training, (3) field coordination, (4) field monitoring, (5) data analytics, (6) feedback loops for decision making, and (7) outcomes assessment. Analyses will include basic descriptive and outlier analyses using machine learning algorithms, which will involve creating features from time-stamps, “don’t know” rates, and skip rates. We will also obtain labeled data from self-filled surveys, and build models using k-folds cross-validation on a training data set using both supervised and unsupervised learning algorithms. Based on these models, results will be fed back to the field through various feedback loops. Results Data collection began in late October 2019 and will span through March 2020. We expect to submit quality assurance results by August 2020. Conclusions Machine learning is underutilized as a tool to improve survey data quality in low resource settings. Study findings are anticipated to improve the overall quality of Kilkari survey data and, in turn, enhance the robustness of the impact evaluation. More broadly, the proposed quality assurance approach has implications for data capture applications used for special surveys as well as in the routine collection of health information by health workers. International Registered Report Identifier (IRRID) DERR1-10.2196/17619

show abstract

The Research Reproducibility Crisis and Economics of Science

Cited by 20 publications

References 79 publications

Replication Redux: The Reproducibility Crisis and the Case of Deworming

Replication Redux: The Reproducibility Crisis and the Case of Deworming

The Open Innovation in Science research field: a collaborative conceptualisation approach

Using Machine Learning to Optimize the Quality of Survey Data: Protocol for a Use Case in India

Contact Info

Product

Resources

About