Recently, in Japan, the percentage of leanness has risen in young women, and the average birth weight has decreased. An increase in the risk of low birth weight has been reported in lean expectant mothers. In this study, we aimed to clarify the relationship between mother's physique at the beginning of pregnancy and the infant's physique, by focusing on sex differences. The participants were 3,722 mothers who attended health checkups for 18-month-old infants in an urban Japanese city. The participants were limited to those with full-term births, thereby excluding the influence of gestational length. A total of 1,287 mothers, with 621 boys and 666 girls, were analyzed. Public health professionals interviewed the mothers, and transferred the required information from their maternity passbooks. We examined the physical characteristics of the mothers and their infants. Partial correlation analysis, adjusted by gestational length and the mother's age at delivery, was applied to study the association between the mother's BMI and the infant's physique at birth. In the primipara group, only the boys showed significant positive correlation between the mother's BMI and the birth weight (P = 0.025) and the Kaup index (P = 0.035). In the pluripara group, only the boys showed significant positive correlation between the mother's BMI and the head circumference (P = 0.035). Thus, mother's physique may have a stronger influence on the physique of male infants, compared to female infants. The growth-promoting effect of the mother's physique is more apparent in the infants born to the pluripara.