Crowdsourcing as a Tool for Research

Abstract

Numerous crowdsourcing platforms are now available to support research as well as commercial goals. However, crowdsourcing is not yet widely adopted by researchers for generating, processing or analyzing research data. This study develops a deeper understanding of the circumstances under which crowdsourcing is a useful, feasible or desirable tool for research, as well as the factors that may influence researchers' decisions around adopting crowdsourcing technology. We conducted semi-structured interviews with 18 researchers in diverse disciplines, spanning the humanities and sciences, to illuminate how research norms and practitioners' dispositions were related to uncertainties around research processes, data, knowledge, delegation and quality. The paper concludes with a discussion of the design implications for future crowdsourcing systems to support research.

The Accuracy of Citizen Science Data: A Quantitative Review

“…, Law et al. ). Some studies evaluating data quality have found volunteer data to be more variable than professionally collected data (Harvey et al.…”

“…Despite the growth in the number of citizen science projects, scientists remain concerned about the accuracy of citizen science data (Danielsen et al 2005, Crall et al 2011, Gardiner et al 2012, Law et al 2017. Some studies evaluating data quality have found volunteer data to be more variable than professionally collected data (Harvey et al 2002, Uychiaoco et al 2005, Belt and Krausman 2012, Moyer-Horner et al 2012 and others that volunteers' performance is comparable to that of professionals or scientists (Hoyer et al 2001, Canfield et al 2002, Oldekop et al 2011.…”

Citizen Science: An Information Quality Research Frontier

“…Consider malfeasance, a persistent concern in open online settings (Law et al 2017), where employing social networking has shown promise for data sabotage detection (Delort et al 2011;Wiggins and He 2016). Research has also considered features of systems that make them more robust to malfeasance (Wiggins and He 2016), as anecdotal evidence has suggested extremely low levels of malicious behavior in citizen science.…”

Crowd-based Multi-Predicate Screening of Papers in Literature Reviews

“…Although not focused on paper screening, we also mention a fascinating analysis by Law and colleagues trying to understand under which conditions do researchers resort to crowdsourcing [22]. Among the many interesting considerations lies the observation that crowdsourcing is viable only if both authors and reviewers find it acceptable.…”