To make accurate causal inferences about social-group inequalities, people must consider structural causes. Structural causes are a distinct type of extrinsic cause—they are stable, interconnected societal forces that systematically advantage some social groups and disadvantage others. We propose a new cognitive framework to specify how people attribute inequality to structural causes. This framework is rooted in counterfactual theories of causal judgment and suggests that people will recognize structural factors as causal when they are perceived as “difference-making” for inequality above and beyond any intrinsic causes. Building on this foundation, our framework makes the following contributions. First, we propose specific types of evidence that support difference-making inferences about structural factors: within-group change (i.e., observing that disadvantaged groups’ outcomes improve under better societal conditions) and well-matched between-group comparisons (i.e., observing that advantaged group members, who have similar baseline traits to the disadvantaged group, experience more favorable societal conditions and life outcomes). Second, we consider contextual, cognitive, and motivational barriers that may complicate the availability and acceptance of this evidence. We conclude by exploring how the framework might be applied in future research examining people’s causal inferences about inequality.