Thermal comfort is a complex issue in the built environment due to the physiological and psychological differences of each individual in a building. There is a growing worry over the environmental implications of energy use as a result of the warming of the global climate and the growth in the number of instances of extreme weather events. Many review articles have been written, but these reviews have focused on a specific aspect of occupant behavior and thermal comfort. To research the trends of thermal comfort and energy, this research adopted mixed reviews, i.e., quantitative and qualitative, to understand the state-of-the-art factors affecting the thermal comfort of occupants concerning energy, different occupant modeling approaches, functions, and limitations. The in-depth qualitative discussion provides deeper insights into the impacts of occupant behaviors, factors affecting thermal comfort, and occupant behavior modeling approaches. This study classified occupant behaviors into five categories: occupant characteristics, perceptions of the occupant, realistic behaviors, heat gain, and occupant interactions with the system. It also went further to classify the factors affecting the thermal comfort of users based on past works of literature. These include structural, environmental, and human factors. It was concluded that factors that have the most significant impact on energy are human, structural, and environmental factors, respectively. In addition, most of the occupant behavior modeling approaches that have been used in past studies have pros and cons and cannot accurately predict human behaviors because they are stochastic. Future research should be conducted on thermal comfort for different building functions by examining the varied activity intensity levels of users, especially in educational or commercial buildings. Additionally, a proper investigation should be carried out on how thermal insulation of structural members influences thermal comfort. These should be compared in two similar buildings to understand occupant behavioral actions and energy consumption.