The aim of this study was to quantify gender-specific facial characteristics in younger and older adults and to determine how aging and body characteristics, such as height and bodymass index (BMI), influence facial sexual dimorphism. Methods The cohort study included 90 younger adults of Caucasian origin (average age of 45 females 23.2 ± 1.9 and 45 males 23.7 ± 2.4 years) and 90 older adults (average age of 49 females 78.1 ± 8.1 and 41 males 74.5 ± 7.7 years). Three-dimensional facial scans were performed with an Artec MHT 3D scanner. The data were analyzed using the software package Rapidform ®. The parameters to evaluate facial symmetry, height, width, profile, facial shape, nose, eyes and mouth characteristics were determined based on 39 facial landmarks. Student's t-test was used to calculate the statistical differences between the genders in the younger and older adults and a multiple-linear-regression analysis was used to evaluate the impact of gender, age, body-mass index and body height. Results We found that the female faces were more symmetrical than the male faces, and this was statistically significant in the older adults. The female facial shape was more rounded and their faces were smaller, after normalizing for body size. The males had wider mouths, longer upper lips, larger noses and more prominent lower foreheads. Surprisingly, we found that all the gender-dependent characteristics were even more pronounced in the older adults. Increased facial asymmetry, decreased facial convexity, increased forehead angle, narrower vermilions and longer inter-eye distances occurred in both genders during aging. An increased BMI was associated with wider faces, more concave facial profiles and wider noses, while greater body height correlated with increased facial heights and wider mouths.