Mammals have evolved sex-specific adaptations to reduce energy usage in times of food scarcity. These adaptations are well described for peripheral tissue, though much less is known about how the energy-expensive brain adapts to food restriction, and how such adaptations differ across the sexes. Here, we examined how food restriction impacts energy usage and function in the primary visual cortex (V1) of adult male and female mice. Molecular analysis and RNA sequencing in V1 revealed that in males, but not in females, food restriction significantly modulated canonical, energy-regulating pathways, including pathways associated with AMP-activated protein kinase (AMPK), peroxisome proliferator-activated receptor alpha (PPARα), mammalian target of rapamycin (mTOR), and oxidative phosphorylation. Moreover, we found that in contrast to males, food restriction in females did not significantly affect V1 ATP usage or visual coding precision (assessed by orientation selectivity). Decreased serum leptin is known to be necessary for triggering energy-saving changes in V1 during food restriction. Consistent with this, we found significantly decreased serum leptin in food-restricted males but no significant change in food-restricted females. Collectively, our findings demonstrate that cortical function and energy usage in female mice are more resilient to food restriction than in males. The neocortex, therefore, contributes to sex-specific, energy-saving adaptations in response to metabolic challenge.