In music listening, limitations on visual experience affect a listener’s abstract information processing and conceptualization of the music. The aim of this study is to examine the differences in emotional responses to music between adults with visual impairment (VI) and adults with normal vision (NV). By using specific, emotion-inducing music reflecting happiness, sadness, anger, and fear, this study considers factors such as music emotion identification, emotional valence, arousal, intensity, and musical preference. A total of 120 participants (60 VI and 60 NV) listened to sixteen 15-second music excerpts and reported which emotion and to what extent they perceived it, using a self-reported music emotion evaluation scale. The results indicated that both of the groups showed high congruence in music emotion identification. However, the VI group showed significantly higher arousal, intensity and preference for sadness, while showing the lowest score for the intensity of fear. The main factor affecting emotion identification was preference for the VI group, and valence for the NV group.