We investigate the thermal Casimir-Polder (CP) potential of 87 Rb atoms in Rydberg nS-states near single-and double-layer graphene. The dependence of the CP potential on parameters such as atom-surface distance, temperature, principal quantum number n and graphene Fermi energy are explored. Through large scale numerical simulations, we show that, in the non-retarded regime, the CP potential is dominated by the non-resonant and evanescent-wave terms which are monotonic, and that, in the retarded regime, the CP potential exhibits spatial oscillations. We identify that the most important contributions to the resonant component of the CP potential come from the nS-nP and nS-(n − 1)P transitions. Scaling of the CP potential as a function of the principal quantum number and temperature is obtained. A heterostructure comprising hexagonal boron nitride layers sandwiched between two graphene layers is also studied. When the boron nitride layer is sufficiently thin, the CP potential can be weakened by changing the Fermi energy of the top graphene layer.Our study provides insights for understanding and controlling CP potentials experienced by Rydberg atoms near single and multi-layer graphene-based van der Waals heterostructures.