Contemporary art is often challenging for the viewer, especially when it violates classic rules of representation. Also, viewers usually have little knowledge about this type of art, making its reception even more difficult. Our main research question was how the cognitive challenge associated with contemporary art affects eye movement. In particular, we aimed to assess the impact on eye movements of (a) object‐related cognitive challenge in terms of image properties (syntactic and semantic violations) and (b) subject‐related cognitive challenge (composite subjective estimate of image inconsistency, ambiguity, and complexity). The eye movements of expert and naive participants were recorded while they freely viewed digital copies of contemporary paintings (four groups of five paintings each, differing in presence of semantic and syntactic violations). We found that neither violations nor art expertise alone predicted eye movements, although perceived, subjectively experienced cognitive challenge did. In particular, subject‐related cognitive challenge was associated with an increase in visual exploration (longer and more numerous fixations, bigger area of exploration, longer viewing time). The roles of object‐related and subject‐related indicators of cognitive challenge in perception of contemporary art are discussed.