In this prospective cohort study, based on 1,505 mother-infant pairs in rural Bangladesh, we evaluated the associations between early-life exposure to arsenic, cadmium, and lead, assessed via concentrations in maternal and child urine, and children's weights and heights up to age 5 years, during the period 2001–2009. Concurrent and prenatal exposures were evaluated using linear regression analysis, while longitudinal exposure was assessed using mixed-effects linear regression. An inverse association was found between children's weight and height, age-adjusted z scores, and growth velocity at age 5 years and concurrent exposure to cadmium and arsenic. In the longitudinal analysis, multivariable-adjusted attributable differences in children's weight at age 5 years were −0.33 kg (95% confidence interval (CI): −0.60, −0.06) for high (≥95th percentile) arsenic exposure and −0.57 kg (95% CI: −0.88, −0.26) for high cadmium exposure, in comparison with children with the lowest exposure (≤5th percentile). Multivariable-adjusted attributable differences in height were −0.50 cm (95% CI: −1.20, 0.21) for high arsenic exposure and −1.6 cm (95% CI: −2.4, −0.77) for high cadmium exposure. The associations were apparent primarily among girls. The negative effects on children's growth at age 5 years attributable to arsenic and cadmium were of similar magnitude to the difference between girls and boys in terms of weight (−0.67 kg, 95% CI: −0.82, −0.53) and height (−1.3 cm, 95% CI: −1.7, −0.89).