Human impact influences morphological variation in animals, as documented in many captive and domestic animal populations. However, there are different levels of human impact, and their influence on the pattern and rate of morphological variation remains unclear. This study contributes to the ongoing debate via the examination of cranial and mandibular shape and size variation and pace of change in Japanese macaques (Macaca fuscata). This species is ideal for tackling such questions because different wild, wild-provisioned, and captive populations have been monitored and collected over seven decades. Linear measurements were taken on 70 skulls from five populations, grouped into three ‘human impact groups’ (wild, wild-provisioned, and captive). This made it possible to investigate the pattern and pace of skull form changes among the human impact groups as well as over time within the populations. It was found that the overall skull shape tends to differ among the human impact groups, with captive macaques having relatively longer rostra than wild ones. Whether these differences are a result of geographic variation or variable human impact, related to nutritional supply and mechanical properties of the diet, is unclear. However, this pattern of directed changes did not seem to hold when the single captive populations were examined in detail. Although environmental conditions have probably been similar for the two examined captive populations (same captive locality), skull shape changes over the first generations in captivity were mostly different. This varying pattern, together with a consistent decrease in body size in the captive populations over generations, points to genetic drift playing a role in shaping skull shape and body size in captivity. In the captive groups investigated here, the rates of change were found to be high compared to literature records from settings featuring different degrees of human impact in different species, although they still lie in the range of field studies in a natural context. This adds to the view that human impact might not necessarily lead to particularly fast rates of change.