For the construction of long and continuous railway lines as well as the replacement of defected rails, rails are joined using flash-butt welding. Under various localized temperatures and thermo-mechanical stresses, a residual stress can develop in the flash-butt welded joint. The residual stress can affect the performance and reliability of the welded rail, particularly in terms of progressive structural damage caused by repeated wheel load. In the present work, the mechanisms of residual stress formation in a flash-butt welded rail and the influence of upsetting force (including its temperature range and magnitude) were investigated using the thermal elastic–plastic finite element analysis. The formation mechanisms of residual stress involved the changes in thermal expansion coefficient, strain, and elastic modulus of the welded joint with respect to temperature. The calculated cooling temperatures and residual stresses in the flash-butt welded joint were in good agreement with the measured results. Compressive residual stresses were observed around the rail head and the rail foot (i.e., approximately −648 MPa at the rail head and −495 MPa at the rail foot), while tensile residual stresses were observed at the rail web (i.e., approximately 165 MPa). It was observed that the investigated compressive upsetting force predominantly induced plastic deformation within the welded joint, resulting in minimal alteration of stress. Consequently, the investigated ranges of upsetting temperature and upsetting forces had an insignificant impact on the formation of residual stress.