Business Not as Usual

Abstract

In January 2014, Psychological Science introduces several significant changes in the journal's publication standards and practices, aimed at enhancing the reporting of research findings and methodology. These changes are incorporated in five initiatives on word limits, evaluation criteria, methodological reports, open practices, and "new" statistics. The scope of these five initiatives is sketched here, along with the reasoning behind them. 1 Revising Word LimitsResearch Articles and Research Reports are the journal's principal platforms for the publication of original empirical research; together they account for more than 80% of all submissions to Psychological Science. Previously, Research Articles and Research Reports were limited to 4,000 and 2,500 words, respectively, and these word limits included all of the main text (introductory sections, Method, Results, and Discussion) along with notes, acknowledgments, and appendices.Going forward, the new limits on Research Articles and Research Reports are 2,000 and 1,000 words, respectively. As before, notes, acknowledgments, and appendices count toward these limits, as do introductory material and Discussion sections. However, the Method and Results sections of a manuscript are excluded from the word limits on Research Articles and Research Reports. The intent here is to allow authors to provide a clear, complete, self-contained description of their studies, which cannot be done with restrictions on Method and Results. But as much as Psychological Science prizes narrative clarity and completeness, so too does it value concision. In almost all cases, a fulsome account of the method and results can be achieved in 2,500 or fewer words for Research Articles and 2,000 or fewer words for Research Reports.

A case study of scholars’ open and sharing practices

“…Wiley and Hilton III (2009, ¶1) have argued that every educational institution must address "openness as a core organizational value if it desires to both remain relevant to its learners and to contribute to the positive advancement of the field of higher education." Today, countless organizationsincluding journals (e.g., Nature Publishing Group, 2013), scholarly societies (e.g., Eich, 2014), funding agencies (e.g., NSERC, 2014), and government-and intergovernment-supported groups (e.g., OLCOS, 2012;OPAL, 2011, UNESCO, 2002-have embraced open practices. These advocates often value openness for its practical benefits and present openness as an instrument for reaching such important goals as reducing costs, increasing impact, and enhancing access.…”

“…Such practices are expected to foster the transparency and visibility of ongoing research projects at all stages of development (Esposito, 2013;Hayden, 2011), including the sharing of data and research materials (Eich, 2014;Nature Publishing Group, 2013). Open practices are often accomplished or accompanied by academic activity on digital environments and online social networks, and these are often considered to be integral to networked scholarship (Veletsianos & Kimmons, 2012a, 2012b.…”

Doctoral thesis: The human fallibility of scientists - dealing with error and bias in academic research

“…Second, where we focused on preregistrations and assessing their quality, it is likewise urgent to develop and assess protocols for using 'open materials', 'open data', and 'open workflows' . These transparent practices have many benefits and are currently gaining traction (e.g., Eich, 2014;Kidwell et al, 2016), but are often insufficiently detailed, documented or structured to allow other researchers to reproduce and replicate results (e.g., reuse of open data requires solid documentation and meta-data; Wicherts, 2017). While we believe all these open practices strengthen research, a lot can still be gained by creating protocols that provide specific, precise, and exhaustive descriptions of materials, data, and workflow.…”

“…Although "planned research" more accurately describes this preregistered research, we will employ the commonly used term "confirmatory research" to describe it. An increasing number of journals now support preregistration for confirmatory research (e.g., Eich, 2014). In addition, over 35 journals now use a format of registered reports (Chambers, 2013) in which the registrations themselves are subject to peer review and revisions before the data collection starts, and the report is accepted for publication regardless of the direction, strength, or statistical significance of the final results.…”

“…kamp, Asendorpf et al, 2013;Bakker, van Dijk, & Wicherts, 2012;Cumming, 2014;Eich, 2014;Funder et al, 2014;John, Loewenstein, & Prelec, 2012;LeBel, Borsboom, Giner-Sorolla, Hasselman, Peters, Ratliff, & Tucker Smith, 2013;Lindsay, 2015;Morey et al, 2016;Nosek et al, 2015;Nosek & Bar-Anan, 2012;Nosek, Spies, & Motyl, 2012;Simmons, Nelson, & Simonsohn, 2011;Simonsohn, Nelson, & Simmons, 2014b;Spellman, 2015;Vazire, 2015Vazire, , 2017Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011;Wagenmakers, Wetzels, Borsboom, van der Maas, & Kievit, 2012;Wicherts, 2011Wicherts, , 2013. Proposed solutions include systemic changes, such as study pre-registration, open peer review, and stricter requirements for documenting, archiving and sharing data (Morey et al, 2016;Nosek et al, 2015;Nosek & Bar-Anan, 2012;.…”