The paper mainly focuses on the study of the effects of Rayleigh damping in the simulations of FRP-concrete bonded joints, thereby proposing an approach to determine the value of its appropriate Rayleigh damping. Specifically, the element tests under Mode I and Mode II fracture modes were first carried out to investigate the effects of the mass proportional Rayleigh damping and the stiffness proportional Rayleigh damping. An FRP-concrete bonded joint is then employed to further investigate the effects of Rayleigh damping on the simulation results under Mode II fracture mode. It is shown that low-frequency vibrations are produced in the simulations of the specimens loaded by Mode I loading and could be damped by the mass proportional Rayleigh damping, while high-frequency vibrations are produced in the simulations of the specimens loaded by Mode II loading and could only be damped by the stiffness proportional Rayleigh damping. It also shows that the stiffness proportional damping is essential to damp out the oscillations in such simulations, thereby improving the convergence. In addition, the procedure proposed in this paper can lead to a proper interval for the value of the stiffness proportional Rayleigh damping, beyond which an unreasonable simulation result may be obtained.