The origin of the temperature divergence between Holocene proxy reconstructions and model simulations remains controversial, but it possibly results from potential biases in the seasonality of reconstructions or in the climate sensitivity of models. Here we present an extensive dataset of Holocene seasonal temperatures reconstructed using 1310 pollen records covering the Northern Hemisphere landmass. Our results indicate that both summer and winter temperatures warmed from the early to mid-Holocene (~11–7 ka BP) and then cooled thereafter, but with significant spatial variability. Strong early Holocene warming trend occurred mainly in Europe, eastern North America and northern Asia, which can be generally captured by model simulations and is likely associated with the retreat of continental ice sheets. The subsequent cooling trend is pervasively recorded except for northern Asia and southeastern North America, which may reflect the cross-seasonal impact of the decreasing summer insolation through climatic feedbacks, but the cooling in winter season is not well reproduced by climate models. Our results challenge the proposal that seasonal biases in proxies are the main origin of model–data discrepancies and highlight the critical impact of insolation and associated feedbacks on temperature changes, which warrant closer attention in future climate modelling.