Urban areas are usually warmer than their surrounding natural areas, an effect known as the urban heat island effect. As such, they are particularly vulnerable to global warming and associated increases in extreme temperatures. Yet ensemble climate‐model projections are generally performed on a scale that is too coarse to represent the evolution of temperatures in cities. Here, for the first time, we combine unprecedented long‐term (35 years) urban climate model integrations at the convection‐permitting scale (2.8 km resolution) with information from an ensemble of general circulation models to assess temperature‐based heat stress for Belgium, a densely populated midlatitude maritime region. We discover that the heat stress increase toward the mid‐21st century is twice as large in cities compared to their surrounding rural areas. The exacerbation is driven by the urban heat island itself, its concurrence with heat waves, and urban expansion. Cities experience a heat stress multiplication by a factor 1.4 and 15 depending on the scenario. Remarkably, the future heat stress surpasses everywhere the urban hot spots of today. Our results demonstrate the need to combine information from climate models, acting on different scales, for climate change risk assessment in heterogeneous regions. Moreover, these results highlight the necessity for adaptation to increasing heat stress, especially in urban areas.