The field of language education has mobilized diversity paradigms during the last several decades. Paradigms, such as world Englishes, English as a lingua franca, and translanguaging, have illuminated how linguistic forms and practices vary across locations, contexts, and individual linguistic repertoires. Although they aim to raise teachers’ and students’ engagement with linguistic heterogeneity, they are largely founded on the postmodern/poststructuralist valorization of linguistic hybridity and fluidity, which tends to neglect language users and thus overlooks the human differences that also inform that heterogeneity. True linguistic diversity and justice can be attained by both problematizing structural obstacles and recognizing that ideologies and structures are entrenched in unequal and unjust relations of power regarding race, gender, class, and sexuality, which influence diverse language users to communicate in certain ways. This conceptual paper problematizes the conventional focus on language in the discourse of linguistic diversity within language education, especially English language teaching, and proposes that we pay greater attention to language users. While recognizing that social justice is not a universal notion, we endorse an antiracist justice-informed contextualized approach to teaching about linguistic diversity by illuminating how diversity and power among language users as well as broader structures impact the nature of communication.