Producing sufficient food to meet rising demand is a precondition for resilience of the global food system in the face of climate and societal changes. Leveraging machine learning techniques, we project total caloric yields, aggregating 100 crops and assuming crop mix adaptation to climate, soil and management conditions. We then estimate terrestrial caloric sufficiency considering population growth, diets, and production driven by climate, agricultural management, and cropland expansion under five climate change and socio-economic scenarios (SSPs). We show that global caloric sufficiency is likely to decrease, despite increased food production, because those gains are outweighed by population growth and higher animal products consumption. Caloric sufficiency decreases for most countries. Among countries facing hunger today, most remain vulnerable, and around 25 countries, mostly in Africa, become more vulnerable. Our results suggest that adapting crop mixes to new climate conditions will likely be insufficient to cope with global changes by 2050.