Extreme-mass-ratio and intermediate-mass-ratio binaries with a millisecond pulsar are gravitational-wave sources that emit also electromagnetic radiation. The millisecond pulsars in these binaries have complex orbital and spin dynamics, which are observable because of spin–orbit and spin–spin coupling (through spin–curvature interaction). The strengths of the couplings generally depends on the mass ratio between the pulsar and the black hole. The narrow mass range of neutron stars gives an advantage in parameter extraction as it greatly reduces the search space, in particular, in the determination of the black-hole mass, in gravitational wave experiments and radio pulsar timing observations. Extreme-mass-ratio and intermediate-mass-ratio binaries with a millisecond pulsar will help to resolve the astrophysical problems, concerning the applicability of the M-σ relation for galactic spheroids extending to the very low-mass galaxies and whether or not low-mass dwarf galaxies and globular clusters would harbour a nuclear intermediate-mass black hole. The high-precision that can be achieved in gravitational wave experiments and radio pulsar timing observations will provide an opportunity to directly detect gravitational clock effects that are arisen from spin couplings. Radio monitoring of the orbital and spin evolution of the millisecond pulsar in an extreme-mass-ratio binary can be used as a bootstrap method for correcting the drifts in the phases in the gravitational waves from the extreme-mass-ratio and intermediate-mass-ratio binaries caused by self-force.