Considering the interaction between a sleeper, ballast layer, and substructure, a three-dimensional coupled discrete-finite element method for a ballasted railway track is proposed in this study. Ballast granules with irregular shapes are constructed using a clump model using the discrete element method. Meanwhile, concrete sleepers, embankments, and foundations are modelled using 20-node hexahedron solid elements using the finite element method. To improve computational efficiency, a GPU-based (Graphics Processing Unit) parallel framework is applied in the discrete element simulation. Additionally, an algorithm containing contact search and transfer parameters at the contact interface of discrete particles and finite elements is developed in the GPU parallel environment accordingly. A benchmark case is selected to verify the accuracy of the coupling algorithm. The dynamic response of the ballasted rail track is analysed under different train speeds and loads. Meanwhile, the dynamic stress on the substructure surface obtained by the established DEM-FEM model is compared with the in situ experimental results. Finally, stress and displacement contours in the cross-section of the model are constructed to further visualise the response of the ballasted railway. This proposed coupling model can provide important insights into high-performance coupling algorithms and the dynamic characteristics of full scale ballasted rail tracks.