We investigate the control of interacting matter through strong coupling to a single electromagnetic mode, such as the photon mode in a Fabry-Perot or split-ring cavity. For this purpose, we analyze the exact effective theory for the collective light-matter hybrid modes of a generic system of N transition dipoles within an interacting solid. The approach allows to predict properties of the coupled light-matter system from the nonlinear response functions of the uncoupled matter "outside the cavity". The limit of large N corresponds to a conventional macroscopic description based on the polarizability of matter. In this limit, the cavity does not affect the static ferroelectric response. Corrections, which are needed to understand finite size systems and to obtain the nonlinear light-matter response, can be obtained from the non-linear susceptibilities of the matter outside the cavity. The theory is benchmarked for the Dicke model, and for a quantum Ising model which serves as a minimal mean-field model for a quantum paraelectric material like SrTiO3.