While the positive effects of music listening have received much attention, insights into the negative effects are rare, such as the experience of boredom. In two consecutive online surveys using qualitative (N = 266) and quantitative methods (N = 719), participants were asked to retrospectively report their experience of boredom while listening to music. The experience of boredom depends on a combination of factors, such as musical features or styles combined with a certain situation, or the dislike of that music. Boredom is also accompanied by feelings of high and low arousal such as annoyance, anger, fatigue, and listlessness. Notably, boredom in the context of music does not only refer to an emotional state but is also used as a disparaging judgment of music. The associated musical features of “boring” music, such as repetition and monotony, go along with a perceived simplicity and uniformity in specific musical styles (e.g., German schlager, pop, EDM, or rap). Overall, the prevalence of boredom during music listening was low in the current sample, where 77% never or rarely experience boredom, but 46% report experiencing it moderately to very severely when it occurs. Participants with higher musical perceptual abilities report experiencing boredom more frequently and intensely, while a higher active engagement with music and higher agreeableness seem to prevent one from “musical” boredom. The results align with typical aspects of general boredom and contribute to the understanding of the variety of factors that compose an averse form of an aesthetic experience with music.