This paper presents a new approach to predict the residual stresses caused by welding in a butt-welded specimen. Based on the developed three-dimensional finite element model, both the process of material filling and the effects of phase transformation are considered through the material properties which depend on both the temperature and temperature history. For simulating the change in volume caused by phase transformation, a linear model is proposed to relate the start temperatures with the peak temperatures. A comparison of the computational and experimental results verifies the applicability of the proposed approach for welding involving material filling and phase transformation. Four models were built to analyze the influence of the material-filling process and phase transformation. The results demonstrated that without the use of material filling process, the residual stress is underestimated. In addition, without phase transformation, the transverse stress in the weld region is underestimated, while the longitudinal and transverse stresses in the weld region and HAZ, respectively, are overestimated.