On making causal claims: A review and recommendations

Abstract

a r t i c l e i n f o a b s t r a c t Social scientists often estimate models from correlational data, where the independent variable has not been exogenously manipulated; they also make implicit or explicit causal claims based on these models. When can these claims be made? We answer this question by first discussing design and estimation conditions under which model estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneity -which includes omitted variables, omitted selection, simultaneity, common-method variance, and measurement error -renders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation could be confounded; these methods include fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of the methodological rigor with which causal claims are being made in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66% and up to 90% of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research.

Nurse staffing and patient outcomes: Strengths and limitations of the evidence to inform policy and practice. A review and discussion paper based on evidence reviewed for the National Institute for Health and Care Excellence Safe Staffing guideline development

“…Only four studies were assessed as relatively strong for both external and internal validity , Patrician et al, 2011, Sales et al, 2008, Spetz et al, 2013. Establishing that presumed cause preceded the presumed effect is a basic requirement for inferring that an observed association between variables is a causal one (Antonakis et al, 2010). However, most studies analysed data in a cross sectional fashion.…”

“…This can bias effect estimates because respondents to a survey tend to provide answers that are consistent in their point of view, leading to halo effects or effects of social desirability (Antonakis et al, 2010). Adverse reports of the practice environment may be related to reports of adverse outcomes not because one causes the other but because both reflect a global negative response.…”

“…It is a potential problem in any observational study and can lead to bias in the estimation of association and hence causal effect (Johnson et al, 2009). While there are several causes of endogeneity (Antonakis et al, 2010) there are some specific patterns of relationship that will predictably lead to endogeneity when assessing the link between staffing and patient outcomes: omitted variables, simultaneity and common-method variance. We address these three below…”

“…However, much of the literature exploring these factors is subject to a third source of endogeneity: common source / common method variance (Antonakis et al, 2010 .…”

It matters how old you feel: Antecedents and performance consequences of average relative subjective age in organizations.

“…This might be particularly worrisome for the core relationship of our study between average relative subjective age and average individual goal accomplishment. Therefore, we performed a further robustness test to control for a potential endogeneity bias, which is a main threat for causality (Antonakis, Bendahan, Jacquart, & Lalive, 2010). For that purpose, we used an instrumental variable approach with two-stage least square (2SLS) estimation techniques (Antonakis et al, 2010) as a final robustness check for our results.…”