Outdoor thermal comfort (OTC) is critical for public health, labor productivity, and human life. Growing extreme heat events caused by climate change have a serious impact on OTCs, especially in urban areas. Quantitatively characterizing and evaluating the spatiotemporal changes in OTCs are essential, and more applications are needed in urban agglomerations. Therefore, taking the Beijing‐Tianjin‐Hebei (BTH) urban agglomeration as the study area, this study aimed to quantitatively assess the summer regional OTC from 1981 to 2020. First, the Universal Thermal Climate Index (UTCI) was used as the indicator of daily thermal stress, and then a Composite Thermal Comfort Score was proposed to evaluate the long‐term, summertime, regional OTC considering the extent, duration, and intensity of daytime and nighttime thermal stress. The results showed that (a) the increase in UTCI (0.32°C/10a at daytime and 0.21°C/10a at nighttime) and heat stress frequency (0.88 at daytime and 0.39 d/10a at nighttime) were manifested over BTH, indicating a worse OTC. Spatial and temporal heterogeneity was also demonstrated. (b) The general OTC showed a decreasing north‐south gradient pattern. At daytime, the northern mountainous zone presented the best OTC, the southern plain zone, especially Hengshui, Langfang, and Cangzhou, showed the worst. At nighttime, the mountain‐plain transition zone showed the best OTC, the northern mountainous zone showed the worst since more cold stress occurred. Our findings will be useful in informing climate change adaptation strategies to ensure urban resilience as extreme heat increases in the context of climate change.