Across vertebrate species, gonadal hormones coordinate physiology with behavior to facilitate social interactions essential for reproduction and survival. In adulthood, these hormones activate neural circuits that regulate behaviors presenting differently in females and males, such as parenting and territorial aggression. Yet long before sex-typical behaviors emerge at puberty, transient hormone production during sensitive periods of neurodevelopment establish the circuits upon which adult hormones act. How transitory waves of early-life hormone signaling exert lasting effects on the brain remains a central question. Here we discuss how perinatal estradiol signaling organizes cellular and molecular sex differences in the rodent brain. We review classic anatomic studies revealing sex differences in cell number, volume, and neuronal projections, and consider how single-cell sequencing methods enable distinction between sex-biased cell-type abundance and gene expression. Finally, we highlight the recent discovery of a gene regulatory program activated by estrogen receptor α (ERα) following the perinatal hormone surge. A subset of this program displays sustained sex-biased gene expression and chromatin accessibility throughout the postnatal sensitive period, demonstrating a bona fide epigenetic mechanism. We propose that ERα-expressing neurons throughout the social behavior network use similar gene regulatory programs to coordinate brain sexual differentiation.