The present study investigated individual differences in how much subjects rely on prior information, such as expectations or knowledge, when faced with perceptual ambiguity. The behavioural performance of forty-four participants was measured on four different visual paradigms (Mooney face recognition, illusory contours, blur detection and representational momentum) in which priors have been shown to affect perception. In addition, questionnaires were used to measure autistic and schizotypal traits in the non-clinical population. We hypothesized that someone who in the face of ambiguous or noisy perceptual input relies heavily on priors, would exhibit this tendency across a variety of tasks. This general pattern would then be reflected in high pairwise correlations between the behavioural measures and an emerging common factor. On the contrary, our results imply that there is no single factor that explains the individual differences present in the aforementioned tasks, as further evidenced by the overall lack of robust correlations between the separate paradigms. Instead, a two-factor structure reflecting differences in the hierarchy of perceptual processing was the best fit for explaining the individual variance in these tasks. This lends support to the notion that mechanisms underlying the effects of priors likely originate from several independent sources and that it is important to consider the role of specific tasks and stimuli more carefully when reporting effects of priors on perception.