We consider an ensemble of ultracold bosonic atoms within a near-planar cavity, driven by a far detuned laser whose phase is modulated at a frequency comparable to the transverse cavity mode spacing. We show that a strong, dispersive atom-photon coupling can be reached for many transverse cavity modes at once. The resulting Floquet polaritons involve a superposition of a set of cavity modes with a density excitation of the atomic cloud. The mutual interactions between these modes lead to distinct avoided crossings between the polaritons. Increasing the laser drive intensity, a low-lying multimode Floquet polariton softens and eventually becomes undamped, corresponding to the transition to a superradiant, self-organized phase. We demonstrate the stability of the stationary state for a broad range of parameters.