Sample Composition Discrepancies in Different Stages of a Probability-based Online Panel

Abstract

We report sample composition discrepancies related to demographic and personality variables occurring in different stages of development of a probability-based online panel. The first stage-selecting eligible participants-produces differences between Internet users and nonusers in age, education, and gender distribution as well as in the personality traits of openness to experience, conscientiousness, and extraversion. The second and third stages of panel development-asking about willingness to

Beyond the Single Organization: Inside Insights From Gaining Access for Large Multiorganization Survey HRD Research

“…They included training providers, accreditation organizations, business forums, professional bodies, industry bodies, and business directories. However, no mention was made of online volunteer panels, despite these being highlighted in the specific literature and offering an alternative approach to recruiting potentially willing respondents (Bosnjak et al, ; Toepoel, ). Given this virtual absence of insights regarding physical access to multiple organizations across both texts and journal articles reviewed, we now offer two inside accounts of the realities of gaining physical access when using Internet questionnaires in survey research.…”

“…Of these, 1,004 had 80% or more of the questions answered, and so can be considered complete responses (American Association for Public Opinion Research, ). As suggested by Bosnjak et al (), demographic data were used to provide indications of possible bias in sample composition. When compared with Department for Business Innovation and Skills () estimates, our respondents overrepresented SMEs from certain U.K. regions (notably London and the South East; Table ), and certain sectors (notably scientific, professional, and technical; Table ).…”

“…Four of these, specified by our client, comprised the SME size (this time between 5 and 249 employees), being based in the United Kingdom, in the private sector, and having been started in 2012 or earlier. Respondents from volunteer panels self‐recruited, first signing up to participate in online surveys regularly and subsequently deciding whether or not to participate in a specific survey, specific mechanisms of self‐selection and incentivization being unknown (Bosnjak et al, ), but meeting industry guidelines (Göritz, ). Because we had not purchased a list of the names or position of potential respondents, we had no control regarding who actually received the questionnaire.…”

“…In particular, they comment that even where lists of e‐mail addresses or online panels of potential respondents are available, response rates are often low and quality variable (Toepoel, ), indicating potential sources of sample composition bias. These include potential respondents’ access to the Internet, willingness to participate in the survey, and willingness to participate in the panel (Bosnjak et al, ). Yet despite issues of access being considered (albeit in varying levels of detail) by the majority of HRD, HRM, and general business and management research methods textbooks, their focus is on research in a single organization and, in particular, building cognitive access to potential respondents, that is, enhancing their willingness to participate to help ensure as high a response rate as possible.…”

Does the Inclusion of Non-Internet Households in a Web Panel Reduce Coverage Bias?

“…In the Netherlands, those without access to the Internet tend to be older, to live alone, and to have migration backgrounds (Leenheer & Scherpenzeel, 2013;van der Laan, 2009). Bosnjak et al (2013) found that Germans without Internet access are less educated and slightly older than those with access; they also detect some personality differences between the two groups. Most studies of the characteristics of the Internet and non-Internet populations have been done in the United States.…”