Using Paradata to Predict and Correct for Panel Attrition

Abstract

This article addresses the questions of whether paradata can help us to improve the models of panel attrition and whether paradata can improve the effectiveness of propensity score weights with respect to reducing attrition biases. The main advantage of paradata is that it is collected as a byproduct of the survey process. However, it is still an open question which paradata can be used to model attrition and to what extent these paradata are correlated with the variables of interest. Our analysis used data from a seven-wave web-based panel survey that had been supplemented by three cross-sectional surveys. This split panel design allowed us to assess the magnitude of attrition bias for a large number of substantive variables. Furthermore, this design enabled us to analyze in detail the effectiveness of propensity score weights. Our results showed that some paradata (e.g., response times and participation history) improved the prediction of panel attrition, whereas others did not. In addition, not all the paradata that increased the model fit resulted in weights that effectively reduced bias. These findings highlight the importance of selecting paradata that are linked to both the survey response process and the variables of interest. This article provides a first contribution to this challenge.

When Students Don't Care: Reexamining International Differences in Achievement and Non-Cognitive Skills

“…The authors in this case found that careless answering patters were related to educational attainment, employment income, a greater likelihood of being employed in a high-skilled job, and self-reported measures of conscientiousness, even after controlling for cognitive ability. Other work in the area of survey research has also found that non response, inconsistent response patterns, and the amount of time respondents spend answering individual items are an indicator of respondent's conscientiousness (See,Jensen & Soland, 2016;Roβmann & Gummer, 2016;Segal, 2012).…”

Integrating Survey Data and Digital Trace Data: Key Issues in Developing an Emerging Field

“…We chose a specific focus on surveys and digital trace data for this special issue. This excludes the combination of survey data with, for example, paradata (e.g., Roßmann & Gummer, 2015), geo data (e.g., Schweers, Kinder-Kurlanda, Müller, & Siegers, 2016), administrative records (e.g., Schnell, 2014), eye tracking (e.g., Vraga, Bode, & Troller-Renfree, 2016), or traditional forms of media content such as television or newspaper coverage (for a review, see de .…”

A Typology of Web Survey Paradata for Assessing Total Survey Error

“…For example, as Kreuter and Olson (2013) summarize, a wealth of studies have examined call records, contact history, and time to completion in studying nonresponse in other modes. However, only a handful of studies (e.g., Crawford, Couper, & Lamias, 2001;Sauermann & Roach, 2013) have investigated similar information for web surveys or utilized prior survey paradata to understand nonresponse in a panel setting (McLauchlan & Schonlau, 2016;Roßmann & Gummer, 2016).…”