Secondhand smoke exposure is a major public health risk that is especially harmful to the developing brain, but it is unclear if early exposure affects brain structure during middle age and older adulthood. Here we analyzed brain MRI data from the UK Biobank in a population-based sample of individuals (ages 44-80) who were exposed (n=2,510) or unexposed (n=6,079) to smoking around birth. We used robust statistical models, including quantile regressions, to test the effect of perinatal smoke exposure (PSE) on cortical surface area (SA), thickness, and subcortical volume. We hypothesized that PSE would be associated with cortical disruption in primary sensory areas compared to unexposed (PSE -) adults. After adjusting for multiple comparisons, SA was significantly lower in the pericalcarine (PCAL), inferior parietal (IPL), and regions of the temporal and frontal cortex of PSE + adults; these abnormalities were associated with increased risk for several diseases, including circulatory and endocrine conditions. Sensitivity analyses conducted in a hold-out group of healthy participants (exposed, n=109, unexposed, n=315) replicated the effect of PSE on SA in the PCAL and IPL. Collectively our results show a negative, long term effect of PSE on sensory cortices that may increase risk for disease later in life.