Human perception differs profoundly between individuals from different cultures. In the present study, we investigated the development of context-sensitive attention (the relative focus on context elements of a visual scene) in a large sample (N = 297) of 5-to 15-year-olds and young adults from rural and urban Brazil, namely from agricultural villages in the Amazon region and the city of São Paulo. We applied several visual tasks which assess context-sensitive attention, including an optical illusion, a picture description, a picture recognition and a facial emotion judgment task. The results revealed that children and adults from the urban sample had a higher level of context-sensitive attention, when compared to children and adults from the rural sample. In particular, participants from São Paulo were more easily deceived by the context elements in an optical illusion task and remembered more context elements in a recognition task than participants from rural Amazon villages. In these two tasks, context-sensitivity increased with age. However, we did not find a cultural difference in the picture description and the facial emotion judgment task. These findings support the idea that visual information processing is highly dependent on the culture-specific learning environments from very early in development. Specifically, they are more consistent with accounts that emphasize the role of the visual environment, than with the social orientation account. However, they also highlight that further research is needed to disentangle the diverse factors that may influence the early development of visual attention, which underlie culture-specific developmental pathways.