Many researchers in impoliteness studies have set themselves the task of determining, amongst other things, (i) what linguistic or non-linguistic phenomena can cause offence, and (ii) why people take offence. However, the reality of interaction clearly shows that, on many occasions, there appears to be a marked dissonance between the speaker and hearer in their evaluations of offensive language, even in locally situated interaction. More research is therefore needed to account for and explain why and how the hearer assigns a particularly offensive meaning to an utterance during the course of an interaction. With this aim, and by drawing on insight from what is referred to as “radical contextualism”, in this study we discuss the possibility of looking at how interactants can arrive at their own (subjective) evaluations of impoliteness in ways that do not match up with the alleged intentions of the so-called offender. Drawing on a number of exchanges that involve such instances of taking offence, we will argue that the taking of offence should best be viewed as a process over which the hearer has a more active control. Accordingly, the paper contributes to current attempts at explaining the variability involved in the taking of offence.