Weighting techniques are employed to generalize results from survey experiments to populations of theoretical and substantive interest. Although weighting is often viewed as a second-order methodological issue, these adjustment methods invoke untestable assumptions about the nature of sample selection and potential heterogeneity in the treatment effect. Therefore, although weighting is a useful technique in estimating population quantities, it can introduce bias and also be used as a researcher degree of freedom. We review survey experiments published in three major journals from 2000–2015 and find that there are no standard operating procedures for weighting survey experiments. We argue that all survey experiments should report the sample average treatment effect (SATE). Researchers seeking to generalize to a broader population can weight to estimate the population average treatment effect (PATE), but should discuss the construction and application of weights in a detailed and transparent manner given the possibility that weighting can introduce bias.

This paper reanalyses data from a recent widely-discussed study reporting that female-authored papers published in top international relations journals received fewer citations than equivalent male-authored papers. The reanalysis indicated that the gender citation gap is largely limited to elite papers, defined either as papers in the right tail of the citation distribution or as papers published in the most familiar journals. Results suggest that the original study's recommendation to consider the gender citation gap in promotion and review requires more data and a better understanding of the factors that influence whether a paper enters the discipline's elite.

Biblical literalists are often described as scientific illiterates, but little if any empirical research has tested this claim. Analysis of a sixteen-item battery from the 2008 US General Social Survey revealed that literalists possess less science knowledge than those with other views of Scripture, but that much of this deficit can be attributed to demographic factors and unequal educational attainment. The marginal direct effect of biblical belief suggests that literalism is not incompatible with knowledge of science and, therefore, the best avenue for increasing science knowledge among literalists may be to foster interest in science and design science courses to attenuate any perceived conflict between science and religion.

Research using symbolic racism has provided evidence that racial bias has widespread social and political impact in the United States, influencing phenomena such as opposition to policies designed to help blacks, disapproval of Barack Obama, and membership in the Tea Party. However, symbolic racism has a racial component and a conservative component, so many researchers have attempted to isolate the racial component of symbolic racism with statistical control; however, the literature lacks guidelines about the effectiveness of such statistical control. To address this shortcoming, I report results from two studies using the 2012 ANES Time Series Study. Study 1 provides guidelines for the effect size necessary to support an inference that variation in a dependent variable is influenced by the racial component of symbolic racism. The nature of this racial component has been inconsistently described in the literature, so Study 2 reports evidence that symbolic racism sometimes predicts black opposition to policies designed to help blacks, which suggests that the characterization of the residual effect of symbolic racism as racial animosity is stronger than warranted by the data. Together, these studies can help researchers better identify when racial bias is an influence and better understand what this influence represents.

No abstract