Impact of different privacy conditions and incentives on survey response rate, participant representativeness, and disclosure of sensitive information: a randomized controlled trial

Abstract

BackgroundAnonymous survey methods appear to promote greater disclosure of sensitive or stigmatizing information compared to non-anonymous methods. Higher disclosure rates have traditionally been interpreted as being more accurate than lower rates. We examined the impact of 3 increasingly private mailed survey conditions—ranging from potentially identifiable to completely anonymous—on survey response and on respondents’ representativeness of the underlying sampling frame, completeness in answering sensitive survey items, and disclosure of sensitive information. We also examined the impact of 2 incentives ($10 versus $20) on these outcomes.MethodsA 3X2 factorial, randomized controlled trial of 324 representatively selected, male Gulf War I era veterans who had applied for United States Department of Veterans Affairs (VA) disability benefits. Men were asked about past sexual assault experiences, childhood abuse, combat, other traumas, mental health symptoms, and sexual orientation. We used a novel technique, the pre-merged questionnaire, to link anonymous responses to administrative data.ResultsResponse rates ranged from 56.0% to 63.3% across privacy conditions (p = 0.49) and from 52.8% to 68.1% across incentives (p = 0.007). Respondents’ characteristics differed by privacy and by incentive assignments, with completely anonymous respondents and $20 respondents appearing least different from their non-respondent counterparts. Survey completeness did not differ by privacy or by incentive. No clear pattern of disclosing sensitive information by privacy condition or by incentive emerged. For example, although all respondents came from the same sampling frame, estimates of sexual abuse ranged from 13.6% to 33.3% across privacy conditions, with the highest estimate coming from the intermediate privacy condition (p = 0.007).ConclusionGreater privacy and larger incentives do not necessarily result in higher disclosure rates of sensitive information than lesser privacy and lower incentives. Furthermore, disclosure of sensitive or stigmatizing information under differing privacy conditions may have less to do with promoting or impeding participants’ “honesty” or “accuracy” than with selectively recruiting or attracting subpopulations that are higher or lower in such experiences. Pre-merged questionnaires bypassed many historical limitations of anonymous surveys and hold promise for exploring non-response issues in future research.

Survey of health care personnel's attitudes toward oral hygiene in long‐term care facilities in Finland

“…A weakness of this study is that the perceptions of nonrespondents, 40% of the target population, might differ from those of respondents, leading to a potential bias in the results. It is known that nonrespondents’ opinions and attitudes may differ significantly from respondents’ views, which should be borne in mind when conclusions are made. Respondents might also have a more positive perception of their work effort than is warranted by their actual contribution, with the answers given accordingly .…”

“…Those who were not at the workplace when the questionnaires were delivered were unable to participate. No rewards or incentives of any kind were offered to respondents …”

Valuing Multiple Benefits, and the Public Perception of SUDS Ponds

“…Whilst attempts were made to retrieve surveys in areas with lower response rates, it is unethical to pester participants to return surveys [24]. Reminder slips with instructions to leave completed surveys in their local library were the chosen solution for this.…”

Antimicrobial use and stewardship practices on Australian beef feedlots

“…In this study, we utilised a mailed questionnaire, and in doing so, identified important shortfalls when using this method to collect data on complex and sensitive issues. For valid inferences to be drawn about the population of interest from the information collected by questionnaire, respondents must be representative of the population from which they are derived, and answers given must be accurate . However, in this study, a major limitation was the low response rate.…”