Solid-state emitters such as epitaxial quantum dots have emerged as a leading platform for efficient, on-demand sources of indistinguishable photons, a key resource for many optical quantum technologies. To maximise performance, these sources normally operate at liquid helium temperatures (~4 K), introducing significant size, weight and power requirements that can be impractical for proposed applications. Here we experimentally resolve the two distinct temperature-dependent phonon interactions that degrade indistinguishability, allowing us to demonstrate that coupling to a photonic nanocavity can greatly improve photon coherence at elevated temperatures compatible with compact cryocoolers. We derive a polaron model that fully captures the temperature-dependent influence of phonons observed in our experiments, providing predictive power to further increase the indistinguishability and operating temperature of future devices through optimised cavity parameters.