We compare relativistic particle integrators commonly used in plasma physics, showing several test cases relevant for astrophysics. Three explicit particle pushers are considered, namely, the Boris, Vay, and HigueraCary schemes. We also present a new relativistic fully implicit particle integrator that is energy conserving. Furthermore, a method based on the relativistic guiding center approximation is included. The algorithms are described such that they can be readily implemented in magnetohydrodynamics codes or Particle-in-Cell codes. Our comparison focuses on the strengths and key features of the particle integrators. We test the conservation of invariants of motion and the accuracy of particle drift dynamics in highly relativistic, mildly relativistic, and non-relativistic settings. The methods are compared in idealized test cases, i.e., without considering feedback onto the electrodynamic fields, collisions, pair creation, or radiation. The test cases include uniform electric and magnetic fields, E B fields, force-free fields, and setups relevant for high-energy astrophysics, e.g., a magnetic mirror, a magnetic dipole, and a magnetic null. These tests have direct relevance for particle acceleration in shocks and in magnetic reconnection.