Comparing and Combining List and Endorsement Experiments: Evidence from Afghanistan

Abstract

List and endorsement experiments are becoming increasingly popular among social scientists as indirect survey techniques for sensitive questions. When studying issues such as racial prejudice and support for militant groups, these survey methodologies may improve the validity of measurements by reducing nonresponse and social desirability biases. We develop a statistical test and multivariate regression models for comparing and combining the results from list and endorsement experiments. We demonstrate that when carefully designed and analyzed, the two survey experiments can produce substantively similar empirical findings. Such agreement is shown to be possible even when these experiments are applied to one of the most challenging research environments: contemporary Afghanistan. We find that both experiments uncover similar patterns of support for the International Security Assistance Force (ISAF) among Pashtun respondents. Our findings suggest that multiple measurement strategies can enhance the credibility of empirical conclusions. Open-source software is available for implementing the proposed methods.

Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence

“…In recent years, the list experiment has grown in popularity as a method for eliciting truthful responses to sensitive questions. Introduced as the “item count technique” by Miller (1984), the procedure has been used to study racial prejudice (Kuklinski, Cobb, and Gilens 1997a; Sniderman and Carmines 1997; Redlawsk, Tolbert, and Franko 2010), drug use (Biemer et al 2005; Coutts and Jann 2011), risky sexual activity (LaBrie and Earleywine 2000; Walsh and Braithwaite 2008), vote buying (Gonzalez-Ocantos et al 2012), and support for military occupation by foreign forces (Blair, Imai, and Lyall 2013). The standard list experiment proceeds by randomly partitioning respondents into control and treatment groups.…”

Using the Predicted Responses from List Experiments as Explanatory Variables in Regression Models

“…A more effective strategy, however, is to incorporate some type of empirical validation within one's research design. For example, researchers can use different indirect questioning techniques, such as endorsement experiments and randomized response methods, and examine whether they provide similar estimates (Blair, Imai, and Lyall 2014). Another strategy is to aggregate individual-level list experiment results and validate them against available true information (Rosenfeld, Imai, and Shapiro 2014).…”

“…Second, statistical methods have been developed for testing whether the key assumptions of list experiments are violated and for reducing the potential biases that arise from violation of such assumptions (Blair and Imai 2012). Third, methodological recommendations have been made to improve the design and validity of list experiments (Blair and Imai 2012;Glynn 2013;Blair, Imai, and Lyall 2014).…”

The differential demand for indirect rule: evidence from the North Caucasus

“…Politically charged and sensitive questions, such as one's attitude toward president Vladimir Putin ("noputin"), or one's attitude toward the local governor ("norep"), exhibit the most missing data. On endorsement and list experiments to elicit more truthful answers to sensitive questions, see Bullock, Imai and Shapiro (2011);Blair, Imai and Lyall (2012 In Figure 1 …”