Exposure to household air pollution (HAP) from cooking with unclean fuels and indoor smoking has become a significant contributor to global mortality and morbidity, especially in low- and middle-income countries such as Nigeria. Growing evidence suggests that exposure to HAP disproportionately affects mothers and children and can increase risks of adverse birth outcomes. We aimed to quantify the association between HAP and adverse birth outcomes of stillbirth, preterm births, and low birth weight while controlling for geographic variability. This study is based on a cross-sectional survey of 127,545 birth records from 41,821 individual women collected as part of the 2018 Nigeria Demographic and Health Survey (NDHS) covering 2013–2018. We developed Bayesian structured additive regression models based on Bayesian splines for adverse birth outcomes. Our model includes the mother’s level and household characteristics while correcting for spatial effects and multiple births per mother. Model parameters and inferences were based on a fully Bayesian approach via Markov Chain Monte Carlo (MCMC) simulations. We observe that unclean fuel is the primary source of cooking for 89.3% of the 41,821 surveyed women in the 2018 NDHS. Of all pregnancies, 14.9% resulted in at least one adverse birth outcome; 14.3% resulted in stillbirth, 7.3% resulted in an underweight birth, and 1% resulted in premature birth. We found that the risk of stillbirth is significantly higher for mothers using unclean cooking fuel. However, exposure to unclean fuel was not significantly associated with low birth weight and preterm birth. Mothers who attained at least primary education had reduced risk of stillbirth, while the risk of stillbirth increased with the increasing age of the mother. Mothers living in the Northern states had a significantly higher risk of adverse births outcomes in 2018. Our results show that decreasing national levels of adverse birth outcomes depends on working toward addressing the disparities between states.