It is projected that, on average, annual temperature will increase between 2 °C to 6 °C under high emission scenarios by the end of the 21st century, with serious consequences in food and nutrition security, especially within semi-arid regions of sub-Saharan Africa. This study aimed to investigate the impact of historical long-term climate (temperature and rainfall) variables on the yield of five major crops viz., sorghum, sesame, cotton, sunflower, and millet in Gedaref state, Sudan over the last 35 years. Mann–Kendall trend analysis was used to determine the existing positive or negative trends in temperature and rainfall, while simple linear regression was used to assess trends in crop yield over time. The first difference approach was used to remove the effect of non-climatic factors on crop yield. On the other hand, the standardized anomaly index was calculated to assess the variability in both rainfall and temperature over the study period (i.e., 35 years). Correlation and multiple linear regression (MLR) analyses were employed to determine the relationships between climatic variables and crops yield. Similarly, a simple linear regression was used to determine the relationship between the length of the rainy season and crop yield. The results showed that the annual maximum temperature (Tmax) increased by 0.03 °C per year between the years 1984 and 2018, while the minimum temperature (Tmin) increased by 0.05 °C per year, leading to a narrow range in diurnal temperature (DTR). In contrast, annual rainfall fluctuated with no evidence of a significant (p > 0.05) increasing or decreasing trend. The yields for all selected crops were negatively correlated with Tmin, Tmax (r ranged between −0.09 and −0.76), and DTR (r ranged between −0.10 and −0.70). However, the annual rainfall had a strong positive correlation with yield of sorghum (r = 0.64), sesame (r = 0.58), and sunflower (r = 0.75). Furthermore, the results showed that a longer rainy season had significant (p < 0.05) direct relationships with the yield of most crops, while Tmax, Tmin, DTR, and amount of rainfall explained more than 50% of the variability in the yield of sorghum (R2 = 0.70), sunflower (R2 = 0.61), and millet (R2 = 0.54). Our results call for increased awareness among different stakeholders and policymakers on the impact of climate change on crop yield, and the need to upscale adaptation measures to mitigate the negative impacts of climate variability and change.