Assessing representativeness of a probability‐based online panel in Germany

Struminskaya, Bella; Kaczmirek, Lars; Schaurer, Ines; Bandilla, Wolfgang

doi:10.1002/9781118763520.ch3

Cited by 28 publications

(20 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As important, it offers data on a number of variables of direct relevance to political and social science researchers and that remain largely unexplored with respect to MTurk. Finally, given completion of the survey via Internet by an independently drawn, population-based sample, we can analyze and compare the data from the 2012 Web ANES sample with data from an MTurk sample while controlling for survey mode (see Struminskaya et al, 2014 andLiu &Wang, 2015 on survey mode effects).…”

Section: Methodsmentioning

confidence: 99%

The Demographic and Political Composition of Mechanical Turk Samples

2016

View full text Add to dashboard Cite

One of the most notable recent developments in survey research is the increased usage of online convenience samples drawn from Amazon's Mechanical Turk (MTurk). While scholars have noted various social and political differences (e.g., age, partisanship) between MTurk and population-based samples, the breadth and depth of these variations remain unclear. We investigate the extent to which MTurk samples differ from population samples, and the underlying nature of these differences. We do so by replicating items from the population-based American National Election Studies (ANES) 2012 Time Series Study in a survey administered to a sample of MTurk respondents. With few exceptions, we not only find that MTurk respondents differ significantly from respondents completing the 2012 ANES via the Web but also that most differences are reduced considerably when controlling for easily measurable sample features. Thus, MTurk respondents do not appear to differ fundamentally from population-based respondents in unmeasurable ways. This suggests that MTurk data can be used to advance research programs, particularly if researchers measure and account for a range of political and demographic variables as needed.

show abstract

Section: Methodsmentioning

confidence: 99%

The Demographic and Political Composition of Mechanical Turk Samples

2016

View full text Add to dashboard Cite

show abstract

“…Respondents were recruited by telephone using the dual-frame approach, which included landline and mobile numbers (more information on recruitment in Struminskaya, Kaczmirek, Schaurer, & Bandilla, 2014; data from the project can be accesed via GESIS Archive: https://dbk.gesis.org/dbksearch/GDES-C2.asp?no=0058&DB=D). The target population were German-speaking adults living in private households in Germany who used the Internet for nonwork-related purposes.…”

Section: Methods Datamentioning

confidence: 99%

Respondent Conditioning in Online Panel Surveys

Struminskaya

2015

Social Science Computer Review

Self Cite

View full text Add to dashboard Cite

In this article, we investigate changes in survey reporting due to prior interviewing. Two field experiments were implemented in a probability-based online panel in which the order of the questionnaires was switched. Although experimental methods for studying panel conditioning are favorable, experiments in longitudinal studies are rare. Studies on conditioning demand additional resources and might influence respondents' answers. Panel conditioning is mostly associated with measurement errors. However, the discussion that sees it exclusively as a negative phenomenon is not comprehensive. Learning the rules of the interview may lead to increases or decreases in data quality (advantageous vs. disadvantageous conditioning). Overall, little evidence of advantageous conditioning and no disadvantageous conditioning is found. Apart from this reassuring finding, this aricle advances the field by using propensity score weighting to account for attrition and other confounding factors and by using paradata to evaluate the plausibility of alternative explanations of panel conditioning.

show abstract

“…For example, a study might be designed as a web-based survey; however, to also reach respondents with no or limited Internet access, this web-based survey might be supplemented by a postal surveyas, for example, in the nationally representative GESIS panel, a mixed-mode survey of the general population in Germany (cf. Struminskaya, Kaczmirek, Schaurer, & Bandilla, 2014). Given the present results, the assessment of sensitive behaviors might be biased in mixed-mode surveys when individuals systematically under-report socially undesirable behaviors in paper-and pencil as compared to computer-assisted survey modes.…”

Section: Implications For Survey Researchmentioning

confidence: 99%

Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis

2014

View full text Add to dashboard Cite

In surveys, individuals tend to misreport behaviors that are in contrast to prevalent social norms or regulations. Several design features of the survey procedure have been suggested to counteract this problem; particularly, computerized surveys are supposed to elicit more truthful responding. This assumption was tested in a meta-analysis of survey experiments reporting 460 effect sizes (total N =125,672). Self-reported prevalence rates of several sensitive behaviors for which motivated misreporting has been frequently observed were compared across self-administered paper-andpencil versus computerized surveys. The results revealed that computerized surveys led to significantly more reporting of socially undesirable behaviors than comparable surveys administered on paper. This effect was strongest for highly sensitive behaviors and surveys administered individually to respondents. Moderator analyses did not identify interviewer effects or benefits of audio-enhanced computer surveys. The meta-analysis highlighted the advantages of computerized survey modes for the assessment of sensitive topics.

show abstract

Assessing representativeness of a probability‐based online panel in Germany

Cited by 28 publications

References 13 publications

The Demographic and Political Composition of Mechanical Turk Samples

The Demographic and Political Composition of Mechanical Turk Samples

Respondent Conditioning in Online Panel Surveys

Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis

Contact Info

Product

Resources

About