The loss of inflectional categories is often thought of as a type of simplification. In this paper we present a survey of phenomena involving the reduction of adjective agreement in Scandinavian, using examples from Norwegian, and discuss their diachronic origins, including a new account of the development of indeclinability in adjectives such as kry ‘proud’. These examples each involve lexically restricted non-canonical inflection – syncretism, defectiveness, overdifferentiation and periphrasis – in particular paradigm cells or syntactic environments. They show that the loss of inflection does not necessarily simplify grammar, and in some cases, can increase grammatical complexity by adding lexical exceptions to general rules. This excludes simplification as the motivation, even if it is the eventual result. We argue from these historical developments that speakers are liable to analyse idiosyncratic patterns of inflection as lexically specified, even where more general (but perhaps more abstract) alternatives are possible. Thus speakers do not always operate with a maximally elegant, reductionist approach to inflection classes.