The coupled nonlinear dynamics of ultracold quantum matter and electromagnetic field modes in an optical resonator exhibits a wealth of intriguing collective phenomena. Here we study a Λ-type, threecomponent Bose-Einstein condensate coupled to four dynamical running-wave modes of a ring cavity, where only two of the modes are externally pumped. However, the unpumped modes play a crucial role in the dynamics of the system due to coherent backscattering of photons. On a mean-field level we identify three fundamentally different steady-state phases with distinct characteristics in the density and spatial spin textures: a combined density and spin-wave, a continuous spin spiral with a homogeneous density, and a spin spiral with a modulated density. The spin-spiral states, which are topological, are intimately related to cavity-induced spin-orbit coupling emerging beyond a critical pump power. The topologically trivial density-wave-spin-wave state has the characteristics of a supersolid with two broken continuous symmetries. The transitions between different phases are either simultaneously topological and first-order, or second-order. The proposed setup allows the simulation of intriguing many-body quantum phenomena by solely tuning the pump amplitudes and frequencies, with the cavity output fields serving as a built-in nondestructive observation tool. New J. Phys. 21 (2019) 013029 S Ostermann et al *