Concerns about the effects of global warming on crop yields have been raised due to stagnating yield progress in recent years. However, an understanding of the effects of changes in temperature and rainfall throughout the crop cycle on historical yield progress is lacking in Europe (EU). In this study, positive wheat, barley, rice, and maize yield progress in the EU (1961–2019) was significant, with rates of 0.05, 0.04, 0.05, and 0.07 Tha−1year−1, for the four crops respectively. Much of this progress has been sustained by Eastern European countries (EE), which had the highest yield progress rates. On average, in the case of wheat and barley, a temperature increase of 1°C in the winter resulted in yields increasing by +0.33 Tha−1 in EE. This was potentially due to decreasing cold damage and improved photosynthesis and vegetative growth, supporting positive yield progress. Recent historical (2001–2019) rates of wheat, barley, rice, and maize yield progress were positive in all EU regions except Western Europe (WE), barley in Southern Europe (SE) and wheat in Northern Europe (NE). Stagnated wheat, barley and rice yields in WE were not explained by temperature or rainfall using direct correlations of observed data. However, May and July temperatures were associated with wheat yields in NE (−0.30 Tha−1°C−1), barley in SE (−0.14 Tha−1°C−1) and maize in WE and SE (−0.42 and −0.39 Tha−1°C−1). With increasing temperatures becoming less than optimal for photosynthesis, reducing grain filling duration and increasing drought episodes, crop yields have stagnated for wheat in NE and barley in SE. With consistent increases in temperature and water evaporative demand expected in the future, the interplay among genetic adaptation, increased crop cycle duration, drought tolerance, sowing dates, smart irrigation and sustainable practices may require thorough regional testing to maximise the yields of wheat, barley and maize in Europe.