Print versus electronic surveys: a comparison of two data collection methodologies

Abstract

This paper compares the responses of consumers who submitted answers to a survey instrument focusing on Internet purchasing patterns both electronically and using traditional paper response methods. We present the results of a controlled experiment within a larger data collection effort. The same survey instrument was completed by 416 Internet customers of a major office supplies company, with approximately 60% receiving the survey in paper form and 40% receiving the electronic version. In order to evaluate the efficacy of electronic surveys relative to traditional, printed surveys we conduct two levels of analysis. On a macro-level, we compare the two groups for similarity in terms of fairly aggregate, coarse data characteristics such as response rates, proportion of missing data, scale means and inter-item reliability. On a more fine-grained, micro-level, we compare the two groups for aspects of data integrity such as the presence of data runs and measurement errors. This deeper, finer-grained analysis allows an examination of the potential benefits and flaws of electronic data collection.Our findings suggest that electronic surveys are generally comparable to print surveys in most respects, but that there are a few key advantages and challenges that researchers should evaluate. Notably, our sample indicates that electronic surveys have fewer missing responses and can be coded/presented in a more flexible manner (namely, contingent coding with different respondents receiving different questions depending on the response to earlier questions) that offers researchers new capabilities.

Profiling bicycle tourists: a case of Croatia

“…An individual approach to each of the chosen respondents ensured a high response rate. Boyer, Olson, Calantone, and Jackson (2002) confirm that electronic surveys have fewer missing responses in comparison with printed surveys.…”

Knowledge Adoption in Online Communities of Practice

“…Online research surveys do not appear to create response bias (Boyer et al 2002;de Leeuw and Nicholls 1996;Tanriverdi 2001). Data collection began four days prior to the beginning of a seven-day national holiday in China when many members planned trips.…”

Comparing Perceived Formal and Informal Learning in Face-to-Face versus Online Environments

“…Previous studies showed that the differences between online and paper-and-pencil forms for the same questionnaire are negligible (Boyer, Olson, Calantone, & Jackson, 2002;Denscombe, 2006;Ferrando & Lornzo-Seva, 2005;Leung & Kember, 2005;Lucia, Herrmann, & Killias, 2007). We used Blau and Caspi's (2008) questionnaire with minor changes made to adapt the questions to the current population.…”