Representativeness in Online Surveys through Stratified Samples

Abstract

Ré suméRepré sentativité des enquê tes en ligne à é chantillons stratifié s: Alors que les enquêtes à échantillons nationaux en face-à-face sont considérées comme représentatives de l'ensemble de la population, les échantillons en ligne sont considérés comme biaisés, notamment en termes d'âge, de sexe et d'éducation. Pour éviter ce biais, les données peuvent être pondérées de façon à se rapprocher d'un échantillon représentatif. Dans le cas d'enquêtes en ligne, les femmes âgées ayant un faible niveau d'éducation reçoivent un poids très élevé et les jeunes gens ayant une bonne éducation un poids très faible. Plutô t que de pondérer les données, nous tirons un échantillon stratifié à partir de plus de 20.000 participants à un panel en ligne. De ce fait, les femmes âgées de niveau d'éducation relativement faible ont une très forte probabilité d'entrer dans l'échantillon, les jeunes gens avec une bonne éducation, une très faible. Pour obtenir un échantillon stratifié selon le sexe  âge  éducation, sans erreur de prise en compte, l'échantillon est limité à un âge allant jusqu'à 49 ans. Pour comparer ces données et celles d'une enquête face-à-face représentative, nous avons utilisé un ensemble de questions tirées de l'Enquête sociale allemande 2002. Nous comparons les résultats des deux études sur la base de plus de 1.100 cas, dont chacun est équivalent en termes d'âge, sexe et éducation. On montre que des échantillons stratifiés en ligne ne sont pas représentatifs de l'ensemble de la population. AbstractWhile nationwide face-to-face samples are considered representative for the entire population, online samples are regarded as biased, especially in terms of age, sex and education. To consider this bias, the data can be weighted to receive a representative sample. In case of online surveys, elderly women with a low level of education receive a very high weight and young men with high education a very low one. Instead of weighting the data, we draw a stratified sample from over 20,000 participants of an online panel. Thereby, elderly women with relatively low formal education had a very high likelihood of entering the sample, young men with high education a very low one. To obtain a stratified sample by sex  age  education without coverage error, the sample is restricted to an age of up to 49 years. To compare these data with representative face-to-face data, we used a set of questions from the German Social Survey 2002. We compare the results of the two studies on the basis of more than 1,100 cases, each of which are equivalent in terms of age, sex and education. It is shown that stratified online samples are not representative for the entire population.

Education for Sustainable Development in Germany: Not Just Desired but Also Effective for Transformative Action

“…Concerning the first challenge, according to Blom et al [58], online studies "that do not account for the non-Internet population are susceptible to coverage bias if this population differs from the Internet population on characteristics that are related to key survey topics" (p. 500). According to another study, the group of offline-households in Germany consists predominantly of older and at the same time less educated persons ( [59], see also [60]). It thereby largely differs from the target sample of this study-young people and teachers, while for teachers, a broad variance of age and of job experience is covered by the study.…”

A study on comparing online, telephone and face to face surveys based on different sampling methods in coffee consumer in Vietnam

“…Moreover, online survey respondents tend to be more politically active, more likely to be earlier adopters of technology, and tend to travel and eat out more than face-to-face survey respondents (Baker et al, 2003). Blasius and Brandt (2010) conducted a stratified online study with 1300 cases in Germany and compared it with a representative face-to-face survey. While both samples were equivalent in terms of age, gender and education, the online sample did not represent the entire population.…”

Mail survey abroad with an alternative web survey

“…These challenges have generated a large number of studies that have compared mail surveys with Web surveys in aspects as diverse as response rates (among others, since 2001, Kaplowitz et al 2004;Schneider et al 2005;Converse et al 2008;Lozar Manfreda et al 2008), coverage (Blasius and Brandt 2010), speed of response (Ilieva et al 2002), quality in the completion of the questionnaire (Fricker et al 2005;Denscombe 2009;Saunders 2012), different level of cooperation of various groups (Diment and Garrett-Jones 2007;Bech and Bo Kristensen 2009), changes in response rates when more than one means of response are available, etc.…”