Scientific Utopia: III. Crowdsourcing Science

Abstract

Most scientific research is conducted by small teams of investigators, who together formulate hypotheses, collect data, conduct analyses, and report novel findings. These teams operate independently, as vertically integrated silos. Here we argue that scientific research that is horizontally distributed can provide substantial complementary value, aiming to maximize available resources, promote inclusiveness and transparency, and increase rigor and reliability. This alternative approach enables researchers to tackle ambitious projects that would not be possible under the standard model. Crowdsourced scientific initiatives vary in terms of the degree of communication between project members, from largely independent work curated by a coordination team to crowd collaboration on shared activities. The potential benefits as well as challenges of large scale collaboration span the entire research process: ideation, study design, data collection, data analysis, reporting, and peer review. Complementing traditional small science with crowdsourced approaches holds the potential to accelerate the progress of science and improve the quality of scientific research.

Open Science 2.0: Large-Scale Collaborative Education Research

“…As discussed elsewhere (Uhlmann et al, 2018), we envision collaborative research serving as a complementary addition to traditional independent research. Substantial exploratory, descriptive, and confirmatory work could still be primarily independently conducted, although all could also be the focus of collaborative research.…”

“…Both StudySwap and PsyAccelerator help crowdsource and coordinate resources so that an individual researcher is not responsible for all aspects of the research cycle (or collecting all data). Broadly, persistent collaboration consortia could help more efficiently allocate existing resources, reduce barriers to entry, and increase inclusivity, while also increasing transparency, rigor, and reliability of the results produced (Uhlmann, et al, 2018). Such an infrastructure may also facilitate answering questions that may otherwise not be possible (e.g., by increasing statistical power, increasing diversity of the sample, ability to assess sub-groups that are often too small).…”

Questionable research practices following pre-registration

“…Now, this may be facilitated by implementing StudySwap, an online collaboration system (Chartier, Riegelman, & McCarthy, 2018). Similarly, it would be effective to use crowdsourcing for the distributed experimentation (Uhlmann, Chartier, Ebersole, Errington, Kidwell, Lai et al, 2018). An advanced pre-registration system would be able to combine such methods to make experiments fair, unbiased, transparent, and highly reproducible.…”

Collaborating to Bring the Credibility Revolution to Giftedness and Talent Development Research

“…The concept of large-scale collaborative research helping improve research quality has been introduced previously (Ioannidis, 2014;Uhlmann et al, 2018), including for education research (Makel et al, 2019), and is already regularly applied in fields like physics where papers can have thousands of authors (e.g., Aad et al, 2012;Abbot et al, 2017;Chatrchyan et al, 2012). Here, we focus specifically on how it can help better evaluate/understand learning contexts for gifted and talented students.…”