In commercial pig farming, an increasing number of low birth weight (LBW) piglets are born, due to selection for large litter sizes. While LBW piglets have a higher risk of pre-weaning mortality, a considerable number of these piglets survive to slaughter age. In humans, LBW is a risk factor for long-term cognitive impairments. In pigs, studies examining the post-weaning effects of LBW on cognition have reported contradictory results. Therefore, the current study aimed to assess the effects of LBW on cognitive development in pigs using an improved study design, by (1) testing a larger sample size than previous studies, (2) assessing acute and chronic stress responses to account for a potential altered stress response in LBW pigs, and (3) testing both female and male pigs to account for potential confounding effects of sex. Learning and memory of 20 LBW pigs and 20 normal birth weight (NBW) pigs, both groups consisting of 10 females and 10 males, were compared using a spatial holeboard task. In this task, pigs had to learn and remember the locations of hidden food rewards. After a pig had successfully acquired the task, it was presented with two successive reversal phases during which it was presented with a new configuration of reward locations. The holeboard allows for simultaneous assessment of working and reference memory, as well as measures of motivation, exploration, and behavioral flexibility. Mixed model ANOVAs revealed a transiently impaired reference memory performance of LBW pigs, implying they had more difficulty learning their reward configuration in the holeboard. Also, LBW piglets showed increased pre-weaning hair cortisol concentrations compared to their NBW siblings. No other effects of LBW were found. Sex had no direct or interaction effects on any measures of holeboard performance or stress. It is possible that the enriched housing conditions applied during our study had an ameliorating effect on our pigs' cognitive development. Overall, our results suggest LBW has a negative effect on post-weaning cognitive performance in pigs. This could have welfare consequences as cognitive skills are required for pigs to learn how to correctly respond to their environment.