The in-plane thermal conductivity of polycrystalline diamond near its nucleation site, which is a key parameter to an efficient integration of diamond in modern high power AlGaN/GaN high electron mobility devices, has been studied. By controlling the lateral grain size evolution through the diamond growth conditions it has been possible to increase the in-plane thermal conductivity of the polycrystalline diamond film for a given thickness. Besides, the in-plane thermal conductivity has been found strongly inhomogeneous across the diamond films, being also possible to control this inhomogeneity by the growth conditions. The experimental results has been explained through a combined effect of the phonon mean free path confinement due the grain size and the quality of the grain/grain interfaces, showing that both effects evolve with the grain expansion and are dependant on the diamond growth conditions. This analysis shows how the thermal transport in the near nucleation region of polycrystalline diamond can be controlled, which ultimately opens the door to create ultra-thin layers with a engineered thermal conductivity, ranging from a few W/mK to a few hundreds of W/mK.