Although research on applying machine learning to the performance of the built environment has been advancing considerably, outdoor environment prediction models still need to be more accurate. In this study, I investigated hybrid-driven methods for developing environmental performance prediction models and studied how machine learning algorithms may interpret spatial information in the context of an environmental performance simulation challenge. The simulation of the Universal Thermal Climate Index (UTCI) for outdoor applications served as an example. Specifically, I designed two different network structures, each with six neural network models. These neural network models were built with various numbers of layers, convolutional kernel sizes, and convolutional kernel layers. As shown by these models’ training results, I investigated the effect of model parameter settings on performance. In addition, I conducted interpretable analysis through the visual observation of hidden internal layers. The use of multilayer and small convolutional kernels, as well as an increase in the amount of training data, may be the reason neural network prediction performance was improved. From the perspective of interpretability analysis, the convolutional layer can more accurately analyze building space problems, and full connection layers focus more on the regression between the spatial features and performance results. This “space analysis → data regression” network structure can be expanded to wind environment forecasting or heat environment in the future.