Peer Review of Datasets: When, Why, and How

Abstract

Peer review holds a central place within the scientific communication system. Traditionally, research quality has been assessed by peer review of journal articles, conference proceedings, and books. There is strong support for the peer review process within the academic community, with scholars contributing peer reviews with little formal reward. Reviewing is seen as a contribution to the community as well as an opportunity to polish and refine understanding of the cutting edge of research. This paper discusses the applicability of the peer review process for assessing and ensuring the quality of datasets. Establishing the quality of datasets is a multifaceted task that encompasses many automated and manual processes. Adding research data into the publication and peer review queues will increase the stress on the scientific publishing system, but if done with forethought will also increase the trustworthiness and value of individual datasets, strengthen the findings based on cited datasets, and increase the transparency and traceability of data and publications. This paper discusses issues related to data peer review—in particular, the peer review processes, needs, and challenges related to the following scenarios: 1) data analyzed in traditional scientific articles, 2) data articles published in traditional scientific journals, 3) data submitted to open access data repositories, and 4) datasets published via articles in data journals.

CMIP6 Data Citation of Evolving Data

“…Additional standardization initiatives investigate the data publishing workflows and develop reference models, like the RDA/WDS Workflows Working Group (Austin et al, 2016). Others focus on data review and its integration in the scholarly publishing processing (Mayernik et al, 2015).…”

Open Data and the Knowledge Society

“…Nevertheless, the articles undergo peer review, as do the underlying data. These types of publications draw attention to the significance of research data as independent publication objects as well as considering their quality and potential for reuse (Mayernik et al 2014). They also help to establish good practices, such as referencing data and making them available through accredited repositories.…”

Key components of data publishing: using current best practices to develop a reference model for data publishing

“…Overall, QA/QC in data publishing is a 'hot-button' topic and is debated heavily and continuously within the community. Mayernik et al describe a range of practice in technical and academic peer review for publishing data [26].…”