Phonon interactions are inevitable in cavity quantum electrodynamical systems based on solid-state emitters or fluorescent molecules, where vibrations of the lattice or chemical bonds couple to the electronic degrees of freedom. Due to the non-Markovian response of the vibrational environment, it remains a significant theoretical challenge to describe such effects in a computationally efficient manner. This is particularly pronounced when the emitter-cavity coupling is comparable with or larger than the typical phonon energy range, and polariton formation coincides with vibrational dressing of the optical transitions. In this paper, we consider four non-Markovian perturbative master equation approaches to describe such dynamics over a broad range of light-matter coupling strengths and compare them with numerically exact reference calculations using a tensor network. The master equations are derived using different basis transformations, and a perturbative expansion in the transformed basis is subsequently introduced and analyzed. We find that two approaches are particularly successful and robust. The first of these is suggested and developed in this paper and is based on a vibrational dressing of the exciton-cavity polaritons. This enables the description of distinct phonon-polariton sidebands that appear when the polariton splitting exceeds the typical phonon frequency scale in the environment. The second approach is based on a variationally optimized polaronic vibrational dressing of the electronic state. Both of these approaches demonstrate good qualitative and quantitative agreement with reference calculations of the emission spectrum and are numerically robust, even at elevated temperatures, where the thermal phonon population is significant.