Considering the significance of proper energy performance analysis of buildings, many recent studies have presented potential applications of machine learning models for predicting buildings’ thermal loads. Some of these models have been built upon optimization algorithms in order to enhance their prediction accuracy. However, due to the importance of time in engineering calculations, the long optimization time of the hybrid models has remained a problem. In this study, a quick optimization algorithm called electromagnetic field optimization (EFO) is presented to deal with this issue. The EFO is combined with a feed-forward artificial neural network (FFANN) to predict the annual thermal energy demand (EDAT) of a residential building based on the building’s characteristics and architecture. A well-known dataset consisting of 11 inputs is used to train and test the proposed model. Additionally, nine conventional FFANNs and several hybrid machine learning are considered benchmark models to evaluate the performance of the EFO-FFANN. According to the results, the calculated mean absolute percentage errors of the EFO-FFANN in the training and testing phases were 2.06% and 1.81%, respectively. The EFO algorithm could improve the prediction accuracy of the conventional FFANNs by around 38%. Hence, the proposed model and its simplified formula can of interest to both civil and energy engineers to do informed decision-making and optimize building energy performance in real-world projects.