Noise annoyance has been often reported as one of the main adverse effects of noise exposure on human health, and there is consensus that it relates to several factors going beyond the mere energy content of the signal. Research has historically focused on a limited set of sound sources (e.g., transport and industrial noise); only more recently is attention being given to more holistic aspects of urban acoustic environments and the role they play in the noise annoyance perceptual construct. This is the main approach promoted in soundscape studies, looking at both wanted and unwanted sounds. In this study, three specific aspects were investigated, namely: (1) the effect of different sound sources combinations, (2) the number of sound sources present in the soundscape, and (3) the presence of individual sound source, on noise annoyance perception. For this purpose, a large-scale online experiment was carried out with 1.2k+ participants, using 2.8k+ audio recordings of complex urban acoustic environments to investigate how they would influence the perceived noise annoyance. Results showed that: (1) the combinations of different sound sources were not important, compared, instead, to the number of sound sources identified in the soundscape recording (regardless of sound sources type); (2) the annoyance ratings expressed a minimum when any two clearly distinguishable sound sources were present in a given urban soundscape; and (3) the presence (either in isolation or combination) of traffic-related sound sources increases noise annoyance, while the presence (either in isolation or combination) of nature-related sound sources decreases noise annoyance.