Taking the simply supported box girder bridge of high-speed railway as an example, the effect of cross-sectional decentralized centre of mass and shear on the spatial beam element stiffness matrix was theoretically derived. Based on the vehicle-bridge coupling vibration analysis method of the railway bridge, an analysis program of vehicle-bridge coupling vibration for the high-speed railway was compiled, and its reliability was verified through an example analysis. On this basis, considering the cross-sectional decentralized centre of mass and shear, the influence factors of vehicle-bridge coupling vibration response were studied, which included the offset distance of the beam section’s mass and shear centre, offset distance of track centreline, vehicle weight, and vehicle speed. The results show that the additional items of the spatial beam element stiffness matrix are generated by the torsion effect when the cross-sectional decentralized centre of mass and shear is considered, and it will affect the lateral and vertical stiffness of the element. The cross-sectional decentralized centre of mass and shear has a significant effect on the lateral dynamic response of the bridge’s mid-span, but the influence on the vertical response of the bridge and the dynamic response of the car body is small. The main influence factors of the lateral dynamic response of the bridge are the vertical offset distance of the beam section’s centre of mass and shear, the lateral offset distance of the track centreline, and the vehicle weight.