Integrating clustering with regression has gained great popularity due to its excellent performance for building energy prediction tasks. However, there is a lack of studies on finding suitable regression models for integrating clustering and the combination of clustering and regression models that can achieve the best performance. Moreover, there is also a lack of studies on the optimal cluster number in the task of short-term forecasting of building energy consumption. In this paper, a comprehensive study is conducted on the integration of clustering and regression, which includes three types of clustering algorithms (K-means, K-medians, and Hierarchical clustering) and four types of representative regression models (Least Absolute Shrinkage and Selection Operator (LASSO), Support Vector Regression (SVR), Artificial Neural Network (ANN), and extreme gradient boosting (XGBoost)). A novel performance evaluation index (PI) dedicated to comparing the performance of two prediction models is proposed, which can comprehensively consider different performance indexes. A larger PI means a larger performance improvement. The results indicate that by integrating clustering, the largest PI for SVR, LASSO, XGBoost, and ANN is 2.41, 1.97, 1.57, and 1.12, respectively. On the other hand, the performance of regression models integrated with clustering algorithms from high to low is XGBoost, SVR, ANN, and LASSO. The results also show that the optimal cluster number determined by clustering evaluation metrics may not be the optimal number for the ensemble model (integration of clustering and regression model).