Small-holder farm households in developing countries mainly depend on rain-fed agriculture activities, thus seasonality affects eating habits and contributes to micronutrient deficiencies. This study assessed women’s dietary diversity score (WDDS) across three agricultural seasons in the Kapchorwa District, Uganda. In each season, 445 women with under five-year-old children from small-holder farm households were interviewed on socio-demographic characteristics as well as dietary practices between May 2016 and January 2017. Linear regression models estimated differences in WDDS across seasons. The mean WDDS at the lean, harvest and post-harvest seasons was 4.33 ± 1.21, 4.63 ± 1.30 and 4.36 ± 1.21, respectively. Mean WDDS was higher in urban regions across all seasons. Women in urban regions generated more income through off-farm activities, had better access to markets and consumed significantly more ‘meat, poultry and fish’ (p < 0.001), dairy products (p < 0.05), ‘vitamin A-rich fruits and vegetables’ (p < 0.001) and ‘other vegetables’ (p < 0.001) during the lean season, whilst rural women ate more dark green leafy vegetables (p < 0.001) during the same period. Poorer households in the rural setting were more likely to be affected by seasonally limited food availability and accessibility. Hence, strengthening the linkages between market participation, agricultural activities, improved food storage and preservation techniques and dietary intake is recommended.