The label "conspiracy theory", while part of everyday media discourse, is considered by many as problematic. In this research, we postulate the existence of diverging social representations associated with this label and that their relative endorsement is a function of preexisting beliefs. To evaluate this possibility, we examine how generic conspiracist beliefs are associated with two forms of rejection of the label: 1) belief in a meta-conspiracy theory, that is, the belief that the label "conspiracy theory" was purposely created by the elites in order to discredit dissent, and 2) a particularist view of conspiracy theories, that is, the idea that "conspiracy theories" are too diverse to draw any generic conclusions regarding their (ir)rationality and that the label is therefore inadequate. Across two studies (Ns = 1297), using Principal Components (Study 1) and Confirmatory Factor Analysis (Study 2), we found that generic conspiracist beliefs and belief in meta-conspiracy theory were tightly related. Moreover, in Study 2, generic conspiracist beliefs were substantially associated with the endorsement of a particularist view of conspiracy theories. Using lexicometric analyses of open-ended questions about the origin and the perceived validity of the label, we identified four main criticisms addressed to the label: historical (real conspiracies happen), conceptual (the label has no clear definition), normative (the label has a negative connotation) and political (the label is weaponized by powerholders). By contrast, acceptance of the label was justified by arguments referring to common psychological mechanisms, as well as similar narrative and argumentative structures. Theoretical and practical implications are discussed. [Manuscript accepted for publication at the International Review of Social Psychology]