Combatting the current global epidemic of obesity requires that people have a realistic understanding of what a healthy body size looks like. This is a particular issue in different population sub-groups, where there may be increased susceptibility to obesity-related diseases. Prior research has been unable to systematically assess body size judgement due to a lack of attention to gender and race; our study aimed to identify the contribution of these factors. Using a data-driven multi-variate decision tree approach, we varied the gender and race of image stimuli used, and included the same diversity among participants. We adopted a condition-rich categorization visual task and presented participants with 120 unique body images. We show that gender and weight categories of the stimuli affect accuracy of body size perception. The decision pattern reveals biases for male bodies, in which participants showed an increasing number of errors from leaner to bigger bodies, particularly under-estimation errors. Participants consistently mis-categorized overweight male bodies as normal weight, while accurately categorizing normal weight. Overweight male bodies are now perceived as part of an expanded normal: the perceptual boundary of normal weight has become wider than the recognized BMI category. For female bodies, another intriguing pattern emerged, in which participants consistently mis-categorized underweight bodies as normal, whilst still accurately categorizing normal female bodies. Underweight female bodies are now in an expanded normal, in opposite direction to that of males. Furthermore, an impact of race type and gender of participants was also observed. Our results demonstrate that perceptual weight categorization is multi-dimensional, such that categorization decisions can be driven by ultiple factors.