Nature can benefit human well-being and cognitive function. Merely watching images of nature compared to urban scenes, which differ in many lower-level processed properties, can have such effects, suggesting that solely the visual input evokes them. In the current series of studies with well-controlled stimuli, we investigated the role of lower- and higher-level processing on restorative effects evoked by nature and urban environments. Therefore, we used not only nature and urban photographs but also 1) versions that lack spatial information but retain certain image properties including those on regularity (i.e., phase-scrambled images), 2) line drawings that contain spatial information and thus allow for higher-level processing while lacking many diagnostic lower-level processed properties, and 3) words that lack any diagnostic image properties but allow higher cognitive processing and provide a mental image of the environment. We examined restorative effects after participants viewed either original, phase-scrambled, or line drawing versions of nature and urban images (Study 1), or nature- and urban-related words (Study 2). Although nature and urban scenes differed in several image properties, their differences did not lead to differences in perceived restoration when presented with phase-scrambled images. However, higher-level processing (i.e., recognizing the presented environment) led to pronounced restoration effects for nature compared to urban stimuli (original images, line drawings, and words). These findings have implications for current theories in the field (i.e., perceptual fluency account, attention restoration theory, and stress recovery theory).