Racism is central to colonial modernity, a global historical project tied to colonialism and imperial governance. Racism and its diverse historical manifestations should in theory be key areas of study for historical sociology: the displacement of populations, the dispossession of lands, genocide, slavery, the coercive exploitation of workers of color, and the creation of a way of seeing people through race. Yet, historical sociological scholarship has to date failed to address how racism and colonialism have structured the modern world. This, we argue, is the result of two problems: First, historical sociology’s ontology and vision of modernity does not see colonialism as central to and constituting the modern world. Second, historical sociology’s Weberian roots produced a methodological approach that detaches theoretical categories from specific historical contexts in the pursuit of generalization. This approach cannot capture the historicity of theory and thus address the subfield’s racial structures of knowledge. As an alternative, we propose that previously silenced anticolonial sociologists provide a different model for historical sociology, one that emphasizes the centrality of colonialism and empire in the constitution of modernity and locates theoretical categories in this historical context. We argue that to overcome the racial underpinning of its knowledge structures, historical sociology has to rebuild itself in this anticolonial mode.