With the rapid development of modern abrasive belt grinding, a novel rail grinding method based on closed-structured belt grinding technology has been proposed in this paper. Correspondingly, a new rail grinding device with the closed abrasive belt was designed according to engineering requirements and constraints. It is equipped with four built-in grinding units aiming to achieve required material removal through lateral, vertical, and rotational movements. Afterward, a corresponding dynamic simulation model for the designed device was established in SIMPACK software to verify its working performance in terms of the four dynamic indices, including lateral vibration acceleration, vertical vibration acceleration, axle transverse force as well as derailment coefficient. The results of those indices both for the linear and curved tracks are within the limits, which have proved the design rationality of the new rail grinding device.