Previous findings suggest that blind listeners are less susceptible to auditory distraction in a verbal serial recall task, compared to sighted individuals. However, it is unclear whether this is due to more selective encoding / filtering of auditory information (e.g., efficient perceptual streaming) or to enhanced attentional control. To test these alternatives, the interference induced by changing-state sound was compared with the disruptive effect of emotional speech prosody (happy, angry, fearful intonations) in blind (n = 17), visually impaired (n = 23), and sighted (n = 89) individuals. While a reduced changing-state effect suggests more efficient perceptual filtering, a reduced emotional prosody effect indicates enhanced attentional control. Blind participants were able to recall more items compared to sighted and visually impaired individuals. Moreover, in sighted and visually impaired participants, the changing-state effect was found to be enhanced with angry prosody, whereas blind individuals were less susceptible to changing-state sound regardless of the prosody. The results also suggest group differences in prosody processing, as both visually impaired and blind participants were better able to ignore fearful speech, compared to other prosodies. The findings suggest that profound visual sensory deprivation leads not only to a capacity increment in verbal short-term memory, but enables listeners to shield memory against auditory distracters, suggesting more efficient perceptual streaming.