This study conceptually replicated and extended Reid, Trofimovich, and O’Brien (2019), who found that native English speakers could be biased positively (or negatively) relative to a control condition in terms of how they rate non-native English speech. Our internet-based study failed to replicate Reid et al. across a wider population sample of “native” speakers (n = 189). Listeners did not change how they rated non-native English speech after social bias orientations and performed similarly across all five measures of speech and across age and race (Asian, Black, and Caucasian). We attribute our results to differences in the methods (in-person vs. online) and/or participants. Of note, roughly one-third of our “native” participants indicated proficiency in languages other than English and residency in 12 different English-speaking countries, despite identifying as a) fluent English speakers who b) used English primarily and c) acquired English before any other language from birth. These screening items taken together qualified “native” participants in line with traditional psycholinguistics research. We conclude that the concept of “nativeness” is tied to culture-specific perspectives surrounding language use. As such, the native/non-native categorical variable simultaneously serves and limits the advancement of psycholinguistics research.