Socially Mediated Internet Surveys: Recruiting Participants for Online Experiments

Abstract

The Socially Mediated Internet Survey (SMIS) method is a cost-effective technique used to obtain web-based, adult samples for experimental research in political science. SMIS engages central figures in online social networks to help recruit participants among visitors to these websites, yielding sizable samples for experimental research. We present data from six samples collected using the SMIS method and compare them to those gathered by other sampling approaches such as Amazon's Mechanical Turk. While not representative of the general adult population, our SMIS samples are significantly more diverse than undergraduate convenience samples, not only demographically but also politically. We discuss the applicability of the method to experimental research and its usefulness for obtaining samples of special, politically relevant subpopulations such as political sophisticates and activists. We argue that the diversity of SMIS samples, along with the ability to capture highly engaged citizens, can circumvent questions about the artificiality of political behavior experiments entirely based on student samples and help to document sources of heterogeneous experimental treatment effects.

Is There a Cost to Convenience? An Experimental Comparison of Data Quality in Laboratory and Online Studies

“…1 The choice of lab versus online administration has important consequences for the burden on subjects and researchers, as well as the quality of the data. From a practical standpoint, online questionnaires can be distributed easily via email, Web sites, or crowd-sourcing platforms such as Mechanical Turk, and participants can complete the survey at a time and place of their choosing (Berinsky et al 2012;Cassese et al 2013). By contrast, laboratory experiments involve greater costs in terms of the administration of the study (e.g., setup, proctoring) and the potential inconvenience for subjects taking the study at a specified time and place.…”

Worldviews, Issue Knowledge, and the Pollution of a Local Science Information Environment

“…Subjects were randomly recruited using Qualtrics panel participants; the sample was stratified to achieve rough representation in terms of gender and age ranges (32.4% of participants were 18-34, 31% were 35-50, and 36.6% were older than 50). Although online surveys have limited generalizability, they have been shown to be useful for generating experimental datasets (Cassese, Huddy, Hartman, Mason, & Weber, 2013) and have been used for similar published research in the area (Kahan, 2013).…”

Crowdsourcing in pediatric transplant studies—An opportunity for reflection on research ethics

“…Supporters of the technology note that it allows researchers to solve many of the problems that have made previous survey studies so difficult. Turkers are generally more diverse and representative of the larger society than convenience samples used in many earlier surveys (eg, undergraduate or medical students) . Recruiting participants is much easier than typical surveys—in the paper in this issue, for example, the researchers achieved their desired sample size of 425 respondents within 40 hours, an almost unthinkably fast method of recruitment compared with more traditional survey methods.…”

“…Turkers are generally more diverse and representative of the larger society than convenience samples used in many earlier surveys (eg, undergraduate or medical students). 8 Recruiting participants is much easier than typical surveys-in the paper in this issue, for example, the researchers achieved their desired sample size of 425 respondents within 40 hours, an almost unthinkably fast method of recruitment compared with more traditional survey methods. The platform allows researchers to meet potential respondents in the online world where they are, instead of increasingly dated methods like phone or mail-in surveys.…”