Fostering unity in diversity while ensuring spaces for disagreement is a key challenge for all liberal democracies with ethnic and religious diversity. Increasing polarization, not least due to the threat of terror attacks, exacerbates this challenge. Drawing on the case of Norway in the aftermath of the 2011 terror attacks motivated by 'Eurabia' sentiments, we find that both consensus and contestation are necessary to counter conflictual polarization. Consensus establishes a necessary common ground for interaction, while contestation permits diverging interpretations to emerge. Working with 21 semi-structured interviews with people in influential roles in Norway, we propose an analytical framework that draws on both political theory and empirically based analyses of interaction in diverse societies. We find that consensus-oriented approaches immediately following terror attacks can build unity and bridge divides across existing ethnic, religious, and political diversity. Over time, however, they may contribute to conflict, as they are perceived to conceal underlying disagreements. Perspectives founded on dualistic contestation can also cultivate conflict if opponents increasingly perceive each other as enemies in a hostile environment. A plurality of contestations, by contrast, can de-escalate conflict and thereby ease renewed cooperation. Thus, our findings point to the need for a perspective that transcends the dualism of ''us'' and ''them'', and acknowledges the plurality of human beings in order to de-escalate the spiral of polarization.