Over the years the field of non-Markovian stochastic processes and anomalous diffusion evolved from a specialized topic to mainstream theory, which transgressed the realms of physics to chemistry, biology and ecology. Numerous phenomenological approaches emerged, which can more or less successfully reproduce or account for experimental observations in condensed matter, biological and/or single-particle systems. However, as far as their predictions are concerned these approaches are not unique, often build on conceptually orthogonal ideas, and are typically employed on an ad hoc basis. It therefore seems timely and desirable to establish a systematic, mathematically unifying and clean approach starting from more finegrained principles. Here we analyze projection-induced ergodic non-Markovian dynamics, both reversible as well as irreversible, using spectral theory. We investigate dynamical correlations between histories of projected and latent observables that give rise to memory in projected dynamics, and rigorously establish conditions under which projected dynamics is Markovian or renewal. A systematic metric is proposed for quantifying the degree of non-Markovianity. As a simple, illustrative but non-trivial example we study single file diffusion in a tilted box, which, for the first time, we solve exactly using the coordinate Bethe ansatz. Our results provide a solid foundation for a deeper and more systematic analysis of projection-induced non-Markovian dynamics and anomalous diffusion.