Effect of Interviewer Experience on Interview Pace and Interviewer Attitudes

Abstract

Traditional statistical analyses of interviewer effects on survey data do not examine whether these effects change over a field period. However, the nature of the survey interview is dynamic. Interviewers' behaviors and perceptions may evolve as they gain experience, thus potentially affecting data quality. This paper looks at how interview length and interviewer evaluations of respondents change over interviewers' workloads. Multilevel models with random interviewer effects are used to account for the clustering of cases within interviewers and individual interviewer characteristics in the 1984, 1988, and 2000 National Election Studies. The 1984 and 1988 NES released sample in four replicates, minimizing the confound between order in an interviewers' workload and sample composition. We find that over the course of the studies, both measures change significantly. Interviewer prior survey experience also was significantly negatively related to the length of the interview. These findings have implications for interviewer training prior to and during studies, as well as suggesting future research to reveal why these behaviors and perceptions change.

The Role Of Interviewer Experience on Acquiescence

“…Second, the sample was not released in random replicates across the study period, confounding sample composition with difficulty in recruiting. Thus, we cannot examine potential changes in these data quality measures over the course of the data collection period (e.g., Olson and Peytchev 2007). Third, although we hypothesize that there are meaningful differences in interviewer behaviors between experienced and inexperienced interviewers, we have no direct measures of these behaviors.…”

“…Second, more experienced interviewers have faster interviews, on average, than do less experienced interviewers (Olson and Peytchev 2007), hypothesized to arise because experienced interviewers place a greater emphasis on productivity (Groves et al 2004, Chapter 9). The increased pace might create an environment where respondents have less time to think through their answers, thereby increasing their likelihood of satisficing (Narayan and Krosnick 1996;Holbrook, Green, and Krosnick 2003).…”

“…New interviewers must become familiar with their role, drawing on behaviors they practiced during training (Olson and Peytchev 2007). More experienced interviewers may not receive training on basic interviewing techniques, relying on previous experience over training (Tarnai and Moore 2008).…”

“…More experienced interviewers may not receive training on basic interviewing techniques, relying on previous experience over training (Tarnai and Moore 2008). Additionally, interviewers who have previously conducted interviews using a particular questionnaire will be more comfortable with the questions, easing and speeding up its administration (Olson and Peytchev 2007). Thus, we expect the largest differences in acquiescence between interviewers with little or no experience and interviewers with some study-specific experience.…”

Is Success in Obtaining Contact and Cooperation Correlated With the Magnitude of Interviewer Variance?

“…We know from the existing literature that interviewer beliefs, attitudes and behavior are predictive of nonresponse (Blom et al, 2010;Durrant and Steele, 2009;Hox and De Leeuw, 2002;Pickery et al, 2001) and measurement error (Singer et al, 1983;Olson and Peytchev, 2007), and we have speculated that traits such as conscientiousness and agreeableness (Digman, 1990) might plausibly account for the patterns of association we have found between these two sources of error. In future research, we will examine which interviewer characteristics are simultaneously predictive of both success in obtaining response and interviewer variance.…”