Background: Optic nerve hypoplasia (ONH), a leading cause of pediatric blindness, is associated with brain malformations and hypopituitarism in the constellation known as septo-optic dysplasia. Neuroimaging is used to anticipate hypopituitarism, but with unconfirmed reliability. We report prospective findings on the association of hypopituitarism with brain malformations. Methods: Children (<24 months) with ONH (n = 146; 87% bilateral) underwent baseline MRI and annual examinations and hormonal testing. Hypopituitarism status at age 5 years was classified. Results: A total of 74% had brain malformation(s). Hypopituitarism (69%) was not associated with brain malformations (p = 0.351); this persisted after adjusting for the laterality of ONH and the timing of MRI (padj = 0.869). No association was noted for absent septum pellucidum (38%; p = 0.073), corpus callosum abnormality (51%; p = 0.625), and major malformations (22%; p = 0.407). A malformation conferred a positive predictive value of 71% (95% CI: 62%, 80%), and a negative predictive value of 37% (95% CI: 22%, 54%). Overall, 10% (n = 15) of the cohort presented with a triad of absent septum pellucidum, corpus callosum abnormality, and other major malformation; only half (n = 8) of these had hypopituitarism. All 13 subjects with pituitary malformations manifested hypopituitarism, conferring predictive values of 100% (positive) and 34% (negative). Conclusions: Hypopituitarism and brain malformations are highly prevalent, but have unrelated associations with ONH. Brain MRI in infants and toddlers with ONH is an unreliable screen for hypopituitarism risk.