Cloud Based Surveys to Assess Patient Perceptions of Health Care: 1000 Respondents in 3 days for US $300

Abstract

BackgroundThere are many challenges in conducting surveys of study participants, including cost, time, and ability to obtain quality and reproducible work. Cloudsourcing (an arrangement where a cloud provider is paid to carry out services that could be provided in-house) has the potential to provide vastly larger, less expensive, and more generalizable survey pools.ObjectiveThe objective of this study is to evaluate, using Amazon's Mechanical Turk (MTurk), a cloud-based workforce to assess patients’ perspectives of health care.MethodsA national online survey posted to Amazon's MTurk consisted of 33 multiple choice and open-ended questions. Continuous attributes were compared using t tests.ResultsWe obtained 1084 responses for a total cost of US $298.10 in less than 3 days with 300 responses in under 6 hours. Of those, 44.74% (485/1084) were male and 54.80% (594/1084) female, representing 49 out of 50 states and aged 18 to 69 years.ConclusionsAmazon’s MTurk is a potentially useful survey method for attaining information regarding public opinions and/or knowledge with the distinct advantage of cost, speed, and a wide and relatively good representation of the general population, in a confidential setting for respondents.

Comparing Amazon’s Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature

“…Crowdsourcing via MTurk can be used as an effective and inexpensive approach for participatory evaluation of medical pictograms. The authors noted that misinterpretations of pictograms made by MTurkers were a result of concepts that were difficult to depict graphically and thus the workers exposed design problems, rather than lack of skill verification, 42 reliable, 27,31 objective, 32 statistically equivalent to data from other samples, 12,22,24,38,49,50 diverse, 19,21,47,49 and viable, 28,36 high quality, 35,42,45 among other strengths, were consistent conclusions in the literature. The weaknesses are dominated by the identified strengths, but important to note.…”

Mapping of Crowdsourcing in Health: Systematic Review (Preprint)

“…Indeed, many health fields have since used crowdsourcing, with 20 medical fields identified in our systematic review compared with 8 fields in the Ranard et al's study [5]. Moreover, crowdsourcing use is still growing, as shown by the 11 articles published in Journal of Medical Internet Research since our last search date, mainly involving a survey task (9/11, 82%) [217][218][219][220][221][222][223][224][225][226][227]. Our study has some limitations.…”

Are tattoos associated with negative health‐related outcomes and risky behaviors?

“…MTurk contains all of the essential elements to conduct survey research, including an integrated compensation system for participants and a streamlined process for study design, participant recruitment, and data collection . A growing body of literature supports and validates the use of this online crowdsourcing service for social science research …”

“…35 A growing body of literature supports and validates the use of this online crowdsourcing service for social science research. 36,37 Studies evaluating MTurk have found respondents to be more demographically diverse than standard college and convenience samples (which are used in the vast majority of the tattoo literature), and although the samples are not necessarily representative of the U.S. population, 38 the data are at least as reliable as those obtained via traditional methods. 35 The present survey was conducted in July 2016 and resulted in a sample of 2,008 adults residing in the U.S.…”