In this paper, we systematically analyze the influence of 5 nm thick metal interlayers inserted at the interface of several sets of different metal-dielectric systems to determine the parameters that most influence interface transport. Our results show that despite the similar Debye temperatures of Al2O3 and AlN substrates, the thermal boundary conductance measured for the Au/Al2O3 system with Ni and Cr interlayers is ~2X and >3X higher than the corresponding Au/AlN system, respectively. We also show that for crystalline SiO2 (quartz) and Al2O3 substrates having highly dissimilar Debye temperature, the measured thermal boundary conductance between Al/Al2O3 and Al/SiO2 are similar in the presence of Ni and Cr interlayers. We suggest that comparing the maximum phonon frequency of the acoustic branches is a better parameter than the Debye temperature to predict the change in the thermal boundary conductance. We show that the electron-phonon coupling of the metallic interlayers also alters the heat transport pathways in a metal-dielectric system in a nontrivial way. Typically, interlayers with large electron-phonon coupling strength can increase the thermal boundary conductance by dragging electrons and phonons into equilibrium quickly. However, our results show that a Ta interlayer, having a high electron-phonon coupling, shows a low thermal boundary conductance due to the poor phonon frequency overlap with the top Al layer. Our experimental work can be interpreted in the context of diffuse mismatch theory and can guide the selection of materials for thermal interface engineering.