Conducting Clinical Research Using Crowdsourced Convenience Samples

Abstract

Crowdsourcing has had a dramatic impact on the speed and scale at which scientific research can be conducted. Clinical scientists have particularly benefited from readily available research study participants and streamlined recruiting and payment systems afforded by Amazon Mechanical Turk (MTurk), a popular labor market for crowdsourcing workers. MTurk has been used in this capacity for more than five years. The popularity and novelty of the platform have spurred numerous methodological investigations, making it the most studied nonprobability sample available to researchers. This article summarizes what is known about MTurk sample composition and data quality with an emphasis on findings relevant to clinical psychological research. It then addresses methodological issues with using MTurk--many of which are common to other nonprobability samples but unfamiliar to clinical science researchers--and suggests concrete steps to avoid these issues or minimize their impact.

Perfectionism Predicts Disordered Gambling Via Financially Focused Self-Concept

“…That said, samples drawn from MTurk are more demographically diverse and representative of the general population than student-based samples are (Buhrmester, Kwang, & Gosling, 2011). Notably, MTurk samples have been shown to yield valid and reliable data for clinical research (for a review, see Chandler & Shapiro 2016), as well as research on disordered gambling (Kim & Hodgins, 2017). Nevertheless, future research should examine whether the findings of the present study can be replicated with a probability sample recruited from the general population.…”

“…Studies indicate that MTurk is a good source of data that is both reliable and valid for conducting clinical research (Chandler & Shapiro, 2016), as well as for research on various addictive behaviours, including gambling (Kim & Hodgins, 2017).…”

How listening to music and engagement with other media provide a sense of belonging: An exploratory study of social surrogacy

“…This distribution is not surprising given that anxiously attached individuals tend to develop relatively strong parasocial relationships with media characters (Cole & Leets, 1999) and therefore might have been more attracted by the topic of the survey. Furthermore, a convenience sample such as the one obtained in this study recruited over social media and the internet has other drawbacks, such as being more highly educated than the general population, being interested in technology, and being drawn from a particularly constrained pool of nationalities (Chandler & Shapiro, 2016).…”

On the Link Between Benevolent Sexism and Orgasm Frequency in Heterosexual Women

“…Participants from the U.S. were recruited using the online survey platform Amazon Mechanical Turk (N = 1,054) (Chandler & Shapiro, 2016). MTurk has been shown to produce results comparable to studies conducted in laboratory settings, with the benefit of a more demographically diverse sample (Buhrmester, Kwang, & Gosling, 2011;Goodman, Cryder, & Cheema, 2013).…”