We investigate the effect of free carrier dynamics on heat transport in bulk crystalline Silicon following femtosecond optical excitation of varying fluences. By taking advantage of the dense 500 MHz standard fill pattern in the PLS-II storage ring, we perform high angular-resolution X-ray diffraction measurements on nanosecond-to-microsecond time-scales with femtometer spatial sensitivity. We find noticeably slowed lattice recovery at increasingly high excitation intensities. Modeling the temporal evolution of lattice displacements due to the migration of the near surface generated heat into the bulk requires reduced thermal diffusion coefficients. We attribute this pump-fluence dependent thermal transport behavior to two separate effects: first, the enhanced nonradiative recombination of free carriers, and, second, reduced size of the effective heat source in the material. These results demonstrate the capability of time-resolved X-ray scattering as an effective means to explore the connection between charge carrier dynamics and macroscopic transport properties.