The effect of questionnaire length on survey response

Abstract

Survey textbooks suggest that long questionnaires should be avoided, and a careful reading of the available empirical evidence confirms the negative effects of substantial length on both response rates and the quality of those responses which are obtained. Data is presented from a lengthy survey in Britain in 1987. Analysis of reasons for nonresponse to this survey suggest that length may indeed have been a significant disincentive to respond for many. However, no effect of length was found on item quality as measured by the number of responses given to open-ended questions. Unexpectedly, the variance in number of responses was greater when the questions were asked later in the questionnaire. The results are interpreted as resulting from the greater power that respondents gain as the survey proceeds. The problemSurvey research is an expensive business; having gone to the expense of tracking respondents down, we often try to get as much from them as possible. While most would concede that increasing length beyond a certain point leads to an unacceptable reduction in both response rates and the quality of the data collected, there is no agreement about the timing of this critical point. In this paper, some findings are reported from a lengthy survey of work attitudes and work histories. We dwell briefly on the reasons for refusal given by non-respondents to this survey, which are commonplace but nonetheless frequently ignored by survey designers. The core of the paper concerns the results of an experiment conducted to test the effect of length of questionnaire on the responses to open-ended questions. The literatureThe message from most survey textbooks is clear: don't make questionnaires too long. Failure to observe this rule will produce a decline in response rates,

The use of cardiac rehabilitation services to aid the recovery of patients with bowel cancer: a pilot randomised controlled trial with embedded feasibility study

“…a non-paper-based questionnaire) was feasible to administer and generally acceptable to participants. Much has been written about the associations between length of questionnaire and response rates; [191][192][193] some participants in our study found the questionnaire to be too long. Nevertheless, 'how long is too long' is likely to be subjective, and face-to-face administered surveys lasting over 1 hour have been found to be acceptable in some studies.…”

The perceived prognosis of endodontic treatment and implant therapy among dental practitioners

“…117 Looking at a study by Burchall and March, they attempted to analyze why people failed to participate in a survey. 118 They found that 61% could have been influenced by the survey length. Love and Turner with the US Census Bureau speculate that increasing length makes a high response rate hard to achieve.…”

Mining Branching Rules from Past Survey Data with an Illustration Using a Geriatric Assessment Survey for Older Adults with Cancer

“…Next, consider the inside out ordering which changes the sequence of the questions numbers to 1,3,5,7,9,10,8,6,4, and 2. The intuition here is that consecutive questions are highly correlated, so separating them in space may lead to more efficient branching rules.…”

“…The response rate of a survey, the extent to which submitted surveys are fully completed, and the reliability of the information provided by respondents each greatly depend on the length of time required to take the survey (see, for example, Burchell and March (1992) [1] and Beebe et al (2010) [2]). Shortening a survey without losing important information is quantitatively feasible when past surveys can be analyzed to develop branching rules for future respondents.…”