The world is faced with significant climate change, rapid urbanization, massive energy consumption, and tremendous pressure to reduce greenhouse gases. Building heating and cooling is one primary source of energy consumption and anthropogenic carbon dioxide emissions. First, this review presents previous studies that estimate the specific amount of climate change impact on building heating and cooling energy consumption, using the statistical method, physical model method, comprehensive assessment model method, and the combination method of statistical and physical model methods. Then, because the heating and cooling degree days indices can simply and reliably indicate the effects of climate on building heating and cooling energy consumption, previous studies were reviewed from the aspects of heating and cooling degree days indices, regional spatial-temporal variations in degree days and related indices, influencing factors of the spatial distributions of degree days, and the impacts of urbanization on degree days. Finally, several potential key issues or research directions were presented according to the research gaps or fields that need to be studied further in the future, such as developing methods to simply and accurately estimate the specified amounts of climate change impact on building cooling and heating energy consumption; using more effective methods to analyze the daytime, nighttime, and all-day spatial-temporal changes in different seasons in the past and future under various environment contexts by considering not only the air temperature but also the relative humidity, solar radiation, population, etc., and further exploring the corresponding more kinds of driving forces, including the various remotely sensed indices, albedo, nighttime light intensity, etc.; estimating the daytime, nighttime, and all-day impacts of urbanization on heating degree days (HDDs), cooling degree days (CDDs), and their sum (HDDs + CDDs) for vast cities in different environmental contexts at the station site, city, regional and global scales; producing and sharing of the related datasets; and analyzing the subsequent effects induced by climate change on the energy consumption for building heating and cooling, etc.