In cities in the global South, internally displaced persons (IDPs) often end up in marginalized places created by uneven processes of urbanization. While IDPs experience similar disadvantages to the urban poor living in these places, they face additional vulnerabilities related to their displacement. Building on insights from urban studies and forced migration studies, we argue in this article that a multidimensional understanding of urban marginality is a useful analytical lens with which to examine the conditions of urban IDPs. Based on multi‐sited ethnographic research in Kersa and Sululta IDP settlements of Ethiopia, this study reveals how IDPs experience similar spatial, social and symbolic marginality in different urban contexts. Our findings show the relational manifestation of segregation, social distance and stigmatization that impede IDPs’ access to urban space and services. This study also highlights how these dimensions of marginality interact and reproduce an additional layer of marginality. Our research suggests the need for inclusive urban governance in which IDPs contribute to and benefit from urbanization as citizens.