The cooling behavior of GaN‐on‐diamond substrate can be enhanced by reducing the thermal boundary resistance (TBR), which is mainly determined by the nature of interlayer. Although SiN film is considered as the primary candidate of dielectric layer, it is still needed to be optimized. In order to facilitate the understanding of the influence of dielectric layer on the TBR of GaN‐on‐Diamond substrate, aluminum nitride (AlN), and silicon nitride (SiN) film were compared systematically, both of which are 100 nm. The time‐domain thermoreflectance (TDTR) measurements, adhesion evaluation, and microstructural analysis methods were adopted to analyse these two interlayers. The results show the TBR of SiN interlayer is as low as 38.5 ± 2.4 m2K GW−1, comparing with the value of 56.4 ± 5.5 m2K GW−1 for AlN interlayer. The difference of TBR between these two interlayers is elucidated by the diamond nucleation density, and the adhesion between the diamond film and GaN substrate, both of which are affected by the surface charge and chemical groups of the dielectric layer.