The sustainability of human societies is contingent upon our ability to accurately predict the effects of future climate change on the global environment and humanity. Wise responses to forthcoming environmental alterations require extensive knowledge from historical precedents. However, in coastal East Asia, a region with a long history of agriculture, it is challenging to obtain paleoenvironmental proxy data without anthropogenic disturbances that can be used to assess the impact of late Holocene climate change on local communities. This study introduces a high-resolution multi-proxy sedimentary record from an isolated crater in Jeju Island, Korea, to elucidate the mechanisms underlying mid-to-late Holocene climate change and its impacts on ancient societies. Our findings suggest that hydroclimate changes were predominantly governed by sea surface temperature fluctuations in the western tropical Pacific, with low-frequency variability in solar activity and a decrease in summer insolation identified as primary drivers of temperature change. Moreover, ancient societies on the Korean peninsula were significantly affected by recurring cooling events, including the 2.8 ka event, 2.3 ka event, Late Antique Little Ice Age, maunder minimum, and others.