The main objective of this study was to examine the consequences of perceived discrimination in people with hearing and visual impairments. Using path analysis, we attempted to validate a multigroup model in which perceived personal discrimination is associated with internalization of stigma, which, in turn, is negatively related to self-esteem; conversely, perceived discrimination against the in-group contributes to enhanced group identification, which promotes the intention to engage in collective action, which, in turn, has beneficial effects on self-esteem. The sample consisted of a total of 200 Spanish-speaking participants, of whom 104 had hearing impairments and 96 had visual impairments. The results showed that the proposed multigroup model fit the data well. For both groups, internalized stigma played a mediating role in the relationship between perceived personal discrimination and self-esteem. However, the pathway from group discrimination to self-esteem was not as clearly supported by the data. The results are interpreted from a psychosocial perspective and may contribute to design interventions aimed at improving the well-being of people with hearing and visual impairments.