Many studies have shown low birth weight is associated with psychopathology later in life, particularly attention-deficit/hyperactivity disorder (ADHD). The association is well-replicated, independent from a variety of potential familial confounds, and follows a dose–response curve (decreasing birth weight linked with increasing odds of disorder). However, the specificity of the association to attention problems is called into question by the extent of comorbidity in ADHD, and recent findings that the association is stronger for autism than ADHD. We test the relative dose–response strength of birth weight on multiple aspects of behavior to explore specificity of the effect to attention problems. We also test recent suggestions that the association between birth weight and attention problems is driven by males. Our sample consisted of 9,076 children aged 9–10 from the United States (Adolescent Brain Cognitive Development study). Outcomes included 9 problem-scales and the total problems scale from the Child Behavior Checklist (CBCL). Attention problems were the most strongly associated with birth weight after controlling for gestational age, potential familial confounds, and multiple testing, supporting the outcome-specificity of this association. Contrary to recent registry-based findings, an association between birth weight and an autism scale was not observed. Sex moderated the effect of birth weight on total problems, attention problems and aggressive behavior such that these inverse associations were strongly driven by males. Our findings have strong implications for sex-specific prediction and etiological models of childhood psychopathology.