Quantum emitters (QEs) coupled to structured baths can localize multiple photons around them and form qubit-photon bound states. In the Markovian or weak coupling regime, the interaction of QEs through these single-photon bound states is known to lead to effective many-body QE Hamiltonians with tuneable but yet perturbative interactions. In this work we study the emergence of such models in the non-Markovian or strong coupling regime in different excitation subspaces. The effective models for the non-Markovian regime with up to three excitations are characterized using analytical methods, uncovering the existence of doublons or triplon states. Furthermore, we provide numerical results for systems with multiple excitations and demonstrate the emergence of polariton models with optically tuneable interactions, whose many-body ground state exhibits a superfluid-Mott insulator transition.intuitive picture [21] is that the single-photon bound state acts as an off-resonant cavity mode that mediates interaction between the QEs. The strength and functional form of these interactions depend on the QE-bath coupling strength, detuning as well as the band-dispersion relation. Although, these interactions can be made relatively strong to overcome other dissipative mechanism, their predicted strength is ultimately limited by the Markovian conditions under which these effective description has been derived.In this work, we study the effective many-body Hamiltonians emerging when QEs couple to structured baths beyond the Markovian limit, and investigate to which point the dipole-dipole description survives in this regime. Furthermore, we study the consequences on the effective QE interactions of the emergence of multiphoton bound states, which were recently predicted in the single QE regime [15, 23-26], but whose impact in the multi QE situation has not yet been fully considered. Our analysis allows us to uncover qualitatively different interaction Hamiltonians, in which multi-photon bound states (doublons/triplons) hop from QE to the other, and analytically characterize them up to three excitations. With more excitations, we numerically characterize the emergent polariton models using density matrix renormalization group (DMRG), and show that their interactions can be controlled optically through the QE-bath interactions, allowing us to probe the quantum phase transition between superfluid and Mott insulator.This manuscript is organized as follows. In section 2, we explain the model that will be used throughout the paper and review the results in the Markovian limit to have them as reference for the next section. In section 3, we study the single excitation subspace, deriving an effective Hamiltonian to describe the dipole-dipole interaction for two and many QEs. In sections 4-6, we study the two-excitation subspace for both the two and many QE regimes. Since the phenomenology in this regime differs significantly from what is expected in a Markovian description, we use section 4 to explain the analytical tools we use to charact...