Emerging evidence points towards a lower quality of life, fragile social relations and suboptimal health behavior and status of residents living in social housing areas characterized by ethnic diversity and socioeconomic deprivation. Community-based health promotion interventions developed in collaboration with the target group and adjusted to the local context can affect the acceptance of and engagement in such interventions. However, few studies have investigated the potential of community-based interventions in deprived social housing areas. This study explores residents’ perspectives on engagement in a community-based health promotion intervention focusing on enhancing social relations. The study builds on qualitative methods including participant observations combined with pre- and post-intervention interviews with a selected group of residents (n = 9). Data were thematically analyzed with focuses on participation in an everyday life context, concepts of othering, and territorial stigmatization. Engagement in the intervention was motivated by the need to establish and enhance social relations, and to explore the world outside the housing area. However, barriers including cultural and language differences among residents, and competing contextual factors, challenged engagement. We conclude that participatory community-based interventions have a potential to enhance social relations in deprived social housing areas. However, adequate support and efforts to overcome the identified barriers are needed.