Physical attractiveness is increasingly framed as a meritocratic good that involves individual benefits, such as higher wages or success in the partner market. Investing in one’s physical appearance is thereby seen as a means to increase one’s human capital. While the positive effects are well documented, its counterpart, the dark side of physical appearance, has received much less attention from social science research. This article sheds light on the negative effects of physical appearance using a theoretical framework based on the cultural sociology of Bourdieu, integrating both structure and agency perspectives. Using data from the German General Social Survey (ALLBUS) from 2014, we demonstrate that unattractiveness is socially stratified by economic, cultural, and social capital. The article highlights the relevance of cultural factors (e.g. forms of cultural capital and cultural practices) for the analysis of the interplay between physical appearance and stratification as well as the relevance of physical appearance for cultural sociology.