Many strongly correlated transition metal insulators are colored, even though they have large fundamental band gaps and no quasi-particle excitations in the visible range. Why such insulators possess the colors they do poses a serious challenge for any many-body theory to reliably pick up the interactions responsible for the color. We pick two archetypal cases as examples: NiO with green color and MnF2 with pink color. To explain the origin of color in these systems, we employ two kinds of advanced ab initio many body Green's function theories to investigate both optical and spin susceptibilities. The first, a perturbative theory based on low-order extensions of the GW approximation, is able to explain the color in NiO, and indeed well describe the dielectric response over the entire frequency spectrum, while the same theory is unable to explain why MnF2 is pink. We show its color originates from higher order spin-flip transitions that modify the optical response. This phenomenon is not captured by low-order perturbation theory, but it is contained in dynamical mean-field theory (DMFT), which has a dynamical spin-flip vertex that contributes to the charge susceptibility. By combining DMFT with an extension of the quasiparticle self-consistent GW approximation to provide a high-fidelity framework for the one-body Green's function, we are able to well describe the peaks in subgap charge susceptibilities in both NiO and MnF2. As a secondary outcome of this work, we establish that the one-particle properties of paramagnetic NiO and MnF2 are both well described by an adequate single Slater-determinant theory and do not require a dynamical vertex.