The Y chromosome is classified into haplogroups (A-T) based on a combination of several DNA polymorphisms. Japanese men are mainly classified into haplogroups C, D, and O, which have been further subdivided. The distribution of Y-chromosome haplogroups varies by ethnicity. The phylogenetic age, origin, and migration also differ. I hypothesized that Y chromosome haplogroups may be associated with height and/or weight at birth. An association analysis of height and weight at birth with Y chromosome haplogroups was performed in 288 Japanese men. Men belonging to haplogroup O1b2 were significantly associated with short stature at birth (beta = −1.88, standard error (SE) = 0.55, P = 0.00076), and those belonging to D1a2a-12f2b were significantly associated with increased birth weight (beta = 174, SE = 64, P = 0.0069). Y chromosome haplogroups are associated with physical birth characteristics in modern Japanese men.