Collections2: Using “Crowdsourcing” within Psychological Research

Abstract

"Crowdsourcing" is a methodological approach in which several researchers coordinate their resources to achieve research goals that would otherwise be difficult to attain individually. This article introduces a Nexus-a collection of empirical and theoretical articles that will be published in Collabra: Psychologythat is intended to encourage more crowdsourced research in psychological science by providing a specific outlet for such projects and by assisting researchers in developing and executing their projects. We describe how individuals can propose and lead a crowdsourced research project, how individuals can contribute to other ongoing projects, and other ways to contribute to this Nexus. Ultimately, we hope this Nexus will contain a set of highly-informative articles that demonstrate the flexibility and range of the types of research questions that can be addressed with crowdsourced research methods. Keywords: methodology; crowdsourcing; collaboration; meta-analysisThe community of research psychologists has access to a large and diverse pool of resources (e.g., time, participants, expertise, geographical locations, etc.) that, collectively, have the potential to produce considerable gains in knowledge, shape public policy, and improve human lives. Despite this potential, these resources may not be collectively used in the most effective manner: Individual researchers often have access to samples that are too small to properly detect the phenomenon of interest, idiosyncrasies in individual samples may not provide evidence about the generalizability of an effect, there may be redundancies in data collection procedures across studies, etc. In addition to being an impediment for scientific progress, these inefficiencies can be viewed as a disservice to the participants who volunteer their time and effort in the belief their data will contribute to reliable and generalizable knowledge, and to the public who entrust researchers to be good stewards of their research resources (e.g., Crutzen, & Peters, 2017). Thus, it is imperative for research psychologists to explore methodological approaches that attempt to address these inefficiencies.One promising methodological approach is "crowdsourced" research, which is an approach that involves several researchers coordinating their resources to achieve goals that would otherwise be difficult to attain individually. For example, several recent, high-profile, large-scale research projects have demonstrated the potential of crowdsourcing research resources to make substantial contributions (e.g., the "Many Labs" projects, Ebersole et al., 2016;Eerland et al., 2016;Klein et al., 2014; Registered Replication Reports [RRR], Alogna et al. 2014;Cheung et al., 2016;Hagger et al., 2016;Wagenmakers et al., 2016; "The Pipeline Project," Schweinsberg et al., 2016; the "ManyBabies" project, Frank et al., 2017; see also Schmalz, 2016). In each of these projects, several research teams each conducted a study (a) following the same methods, (b) at different locations and with differe...

PsychOpen CAMA: Publication of community‐augmented meta‐analyses in psychology

“…The continuous maintenance of a CAMA repository is both time‐ and labor‐intensive. The workload can be reduced via crowdsourcing, 36 if the research community is willing and able to provide relevant data, at best in the desired format. To support users in the submission of data, we plan to use the submission assistant of PsychArchives ( https://www.psycharchives.org/ ).…”

Open Science in Applied Linguistics: An Introduction to Metascience

“…Instead, data collection is crowdsourced from different labs who then combine their data and publish it all in one report. For example, the journal Collabra: Psychology has recently introduced a section called "Nexus" dedicated to crowdsourced research (McCarthy & Chartier, 2017). Inclusion in this collection is not determined by substantive topic as is typically the case in special issues, but by adopting a multi-lab approach to address an issue.…”

The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network

“…Feedback from the review process is given to all proposing authors. Proposing authors whose projects are not selected may be encouraged to revise their protocols or use another network of team-based psychology researchers (e.g., StudySwap; McCarthy & Chartier, 2017), depending on the feedback received during the review process.…”