Prevention measures aimed at combating COVID-19 pandemic strongly impact several aspects of social life. In particular, interpersonal perception is affected as a function of whether the persons perceived wear or not face masks. In two experimental studies, we here explored whether people rely on the presence vs. absence of face masks when encoding information in memory about other individuals. In a memory confusion paradigm, participants were initially presented with individuals either wearing a face mask or not, each conveying a series of sentences. Next, participants were probed about the identity of the speaker of each sentence. Results showed that it was more likely to erroneously attribute a sentence to a speaker who also was wearing a face mask (or not) as the original speaker, demonstrating that the cue about wearing or not a face mask was spontaneously used to encode information. Study 2 ruled out an alternative explanation based on perceptual processes, suggesting that face masks represent meaningful social objects. Overall, it emerged that participants spontaneously categorize others as a function of whether they wear a mask or not. Findings also confirmed previous research evidence about the more positive evaluation of mask wearers as compared to non-wearers, and the overall detrimental impact that face masks may have on the correct identification of social targets.