Developing Standards for Post-Hoc Weighting in Population-Based Survey Experiments

Franco, Annie; Malhotra, Neil; Simonovits, Gábor; Zigerell, L. J.

doi:10.1017/xps.2017.2

Cited by 74 publications

(52 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Consistent with Franco, Malhotra, Simonovits, and Zigerell (), survey weights are not employed in the findings presented below; poststratification‐weighted results are robust to these findings.…”

supporting

confidence: 68%

Partisanship, Political Knowledge, and the Dunning‐Kruger Effect

Anson

2018

Political Psychology

View full text Add to dashboard Cite

A widely cited finding in social psychology holds that individuals with low levels of competence will judge themselves to be higher achieving than they really are. In the present study, I examine how the so-called "Dunning-Kruger effect" conditions citizens' perceptions of political knowledgeability. While low performers on a political knowledge task are expected to engage in overconfident self-placement and self-assessment when reflecting on their performance, I also expect the increased salience of partisan identities to exacerbate this phenomenon due to the effects of directional motivated reasoning. Survey experimental results confirm the Dunning-Kruger effect in the realm of political knowledge. They also show that individuals with moderately low political expertise rate themselves as increasingly politically knowledgeable when partisan identities are made salient. This below-average group is also likely to rely on partisan source cues to evaluate the political knowledge of peers. In a concluding section, I comment on the meaning of these findings for contemporary debates about rational ignorance, motivated reasoning, and political polarization.

show abstract

supporting

confidence: 68%

Partisanship, Political Knowledge, and the Dunning‐Kruger Effect

Anson

2018

Political Psychology

View full text Add to dashboard Cite

show abstract

“…The results reported below are unweighted, as the estimands of interest are sample average treatment effects (Franco et al. 2016).…”

Section: Methodsmentioning

confidence: 99%

Labels vs. Pictures: Treatment-Mode Effects in Experiments About Discrimination

Abrajano

Elmendorf

Quinn³

2018

Polit. Anal.

View full text Add to dashboard Cite

Does treatment mode matter in studies of the effects of candidate race or ethnicity on voting decisions? The assumption implicit in most such work is that such treatment mode differences are either small and/or theoretically well understood, so that the choice of how to signal the race of a candidate is largely one of convenience. But this assumption remains untested. Using a nationally representative sample of white voting-age citizens and a modified conjoint design, we evaluate whether signaling candidate ethnicity with ethnic labels and names results in different effects than signaling candidate ethnicity with ethnically identifiable photos and names. Our results provide strong evidence that treatment-mode effects are substantively large and statistically significant. Further, these treatment-mode effects are not consistent with extant theoretical accounts. These results highlight the need for additional theoretical and empirical work on race/ethnicity treatment-mode effects.

show abstract

“…However, practical advice remains limited in the literature and uncertainty persists among scholars regarding the role of weights that capture differing probabilities of eventual inclusion across units in the analysis of survey experiments (Franco et al. 2017). Should they be used or ignored?…”

Section: Introductionmentioning

confidence: 99%

Worth Weighting? How to Think About and Use Weights in Survey Experiments

Sekhon²,

et al. 2018

View full text Add to dashboard Cite

The popularity of online surveys has increased the prominence of using weights that capture units' probabilities of inclusion for claims of representativeness. Yet, much uncertainty remains regarding how these weights should be employed in analysis of survey experiments: Should they be used or ignored? If they are used, which estimators are preferred? We offer practical advice, rooted in the Neyman-Rubin model, for researchers producing and working with survey experimental data. We examine simple, efficient estimators for analyzing these data, and give formulae for their biases and variances. We provide simulations that examine these estimators as well as real examples from experiments administered online through YouGov. We find that for examining the existence of population treatment effects using highquality, broadly representative samples recruited by top online survey firms, sample quantities, which do not rely on weights, are often sufficient. We found that Sample Average Treatment Effect (SATE) estimates did not appear to differ substantially from their weighted counterparts, and they avoided the substantial loss of statistical power that accompanies weighting. When precise estimates of Population Average Treatment Effects (PATE) are essential, we analytically show post-stratifying on survey weights and/or covariates highly correlated with the outcome to be a conservative choice. While we show these substantial gains in simulations, we find limited evidence of them in practice.

show abstract

Developing Standards for Post-Hoc Weighting in Population-Based Survey Experiments

Cited by 74 publications

References 28 publications

Partisanship, Political Knowledge, and the Dunning‐Kruger Effect

Partisanship, Political Knowledge, and the Dunning‐Kruger Effect

Labels vs. Pictures: Treatment-Mode Effects in Experiments About Discrimination

Worth Weighting? How to Think About and Use Weights in Survey Experiments

Contact Info

Product

Resources

About