Vertebrate‐mediated seed dispersal is a common attribute of many living plants, and variation in the size and abundance of fleshy diaspores is influenced by regional climate and by the nature of vertebrate seed dispersers among present‐day floras. However, potential drivers of large‐scale variation in the abundance and size distributions of fleshy diaspores through geological time, and the importance of geographic variation, are incompletely known. This knowledge gap is important because fleshy diaspores are a key mechanism of energy transfer from photosynthesis to animals and may in part explain the diversification of major groups within birds and mammals. Various hypotheses have been proposed to explain variation in the abundance and size distribution of fleshy diaspores through time, including plant–frugivore co‐evolution, angiosperm diversification, and changes in vegetational structure and climate. We present a new data set of more than 800 georeferenced fossil diaspore occurrences spanning the Triassic–Oligocene, across low to mid‐ to high palaeolatitudes. We use this to quantify patterns of long‐term change in fleshy diaspores, examining the timing and geographical context of important shifts as a test of the potential evolutionary and climatic explanations. We find that the fleshy fruit sizes of angiosperms increased for much of the Cretaceous, during the early diversification of angiosperms from herbaceous ancestors with small fruits. Nevertheless, this did not cause a substantial net change in the fleshy diaspore size distributions across seed plants, because gymnosperms had achieved a similar size distribution by at least the Late Triassic. Furthermore, gymnosperm‐dominated Mesozoic ecosystems were mostly open, and harboured low proportions of specialised frugivores until the latest Cretaceous, suggesting that changes in vegetation structure and plant–frugivore co‐evolution were probably not important drivers of fleshy diaspore size distributions over long timescales. Instead, fleshy diaspore size distributions may be largely constrained by physical or life‐history limits that are shared among groups and diversify as a plant group expands into different growth forms/sizes, habitats, and climate regimes. Mesozoic gymnosperm floras had a low abundance of fleshy diaspores (<50% fleshy diaspore taxa), that was surpassed by some low‐latitude angiosperm floras in the Cretaceous. Eocene angiosperm floras show a mid‐ to high latitude peak in fleshy fruit abundance, with very high proportions of fleshy fruits that even exceed those seen at low latitudes both in the Eocene and today. Mid‐ to high latitude proportions of fleshy fruits declined substantially over the Eocene–Oligocene transition, resulting in a shift to more modern‐like geographic distributions with the highest proportion of fleshy fruits occurring in low‐latitude tropical assemblages. This shift was coincident with global cooling and the onset of Southern Hemisphere glaciation, suggesting that rapid cooling at mid‐ and high latitudes caused a decrease in availability of the climate conditions most favourable for fleshy fruits in angiosperms. Future research could be focused on examining the environmental niches of modern fleshy fruits, and the potential effects of climate change on fleshy fruit and frugivore diversity.