Employing internal quantum systems as reference frames is a crucial concept in quantum gravity, gauge theories, and quantum foundations whenever external relata are unavailable. In this work, we give a comprehensive and self-contained treatment of such quantum reference frames (QRFs) for the case when the underlying configuration space is a finite Abelian group, significantly extending our previous work [M. Krumm, P. A. Höhn, and M. P. Müller, Quantum 5, 530 (2021)]. The simplicity of this setup admits a fully rigorous quantum information–theoretic analysis, while maintaining sufficient structure for exploring many of the conceptual and structural questions also pertinent to more complicated setups. We exploit this to derive several important structures of constraint quantization with quantum information–theoretic methods and to reveal the relation between different approaches to QRF covariance. In particular, we characterize the “physical Hilbert space”—the arena of the “perspective-neutral” approach—as the maximal subspace that admits frame-independent descriptions of purifications of states. We then demonstrate the kinematical equivalence and, surprising, dynamical inequivalence of the “perspective-neutral” and the “alignability” approach to QRFs. While the former admits unitaries generating transitions between arbitrary subsystem relations, the latter, remarkably, admits no such dynamics when requiring symmetry-preservation. We illustrate these findings by example of interacting discrete particles, including how dynamics can be described “relative to one of the subystems.”