During pregnancy and infancy, the human brain is growing extremely fast; the brain volume increases significantly, reaching 36, 72, and 83% of the volume of adults at 2–4 weeks, 1 year, and 2 years of age, respectively, which is essential to establish the neuronal networks and capacity for the development of cognitive, motor, social, and emotional skills that will be continually refined throughout childhood and adulthood. Such dramatic changes in brain structure and function are associated with very large energetic demands exceeding by far those of other organs of the body. It has been estimated that during childhood the brain may account for up to 60% of the body basal energetic requirements. While the main source of energy for the adult brain is glucose, it appears that it is not sufficient to sustain the dramatic metabolic demands of the brain during its development. Recently, it has been proposed that this energetic challenge is solved by the ability of the brain to use ketone bodies (KBs), produced from fatty acid oxidation, as a complement source of energy. Here, we first describe the main cellular and physiological processes that drive brain development along time and how different brain metabolic pathways are engaged to support them. It has been assumed that the majority of energetic substrates are used to support neuronal activity and signal transmission. We discuss how glucose and KBs are metabolized to provide the carbon backbones used to synthesize lipids, nucleic acid, and cholesterol, which are indispensable building blocks of neuronal cell proliferation and are also used to establish and refine brain connectivity through synapse formation/elimination and myelination. We conclude that glucose and KBs are not only important to support the energy needs of the brain under development, but they are also essential substrates for the biosynthesis of macromolecules underlying structural brain growth and reorganization. We emphasize that glucose and fatty acids supporting the production of KBs are provided in complex food matrices, such as breast milk, and understanding how their availability impacts the brain will be key to promote adequate nutrition to support brain metabolism and, therefore, optimal brain development.