During severe nuclear accidents, radioactive materials are expected to be released into the atmosphere. Estimating the source term plays a significant role in assessing the consequences of an accident to assist in actioning a proper emergency response. However, it is difficult to obtain information on the source term directly through the instruments in the reactor because of the unpredictable conditions induced by the accident. In this study, a deep learning-based method to estimate the source term with field environmental monitoring data, which utilizes the bagging method to fuse models based on the temporal convolutional network (TCN) and two-dimensional convolutional neural network (2D-CNN), was developed. To reduce the complexity of the model, the particle swarm optimization algorithm was used to optimize the parameters in the fusion model. Seven typical radionuclides (Kr-88, I-131, Te-132, Xe-133, Cs-137, Ba-140, and Ce-144) were set as mixed source terms, and the International Radiological Assessment System was used to generate model training data. The results indicated that the average prediction error of the fusion model for the seven nuclides in the test set was less than 10%, which significantly improved the estimation accuracy compared with the results obtained by TCN or 2D-CNN. Noise analysis revealed the fusion model to be robust, having potential applicability toward more complex nuclear accident scenarios.