We present the formulation of a version of Lorentz covariant quantum mechanics based on a group theoretical construction from a Heisenberg–Weyl symmetry with position and momentum operators transforming as Minkowski four-vectors. The basic representation is identified as a coherent state representation, essentially an irreducible component of the regular representation, with the matching representation of an extension of the group C*-algebra giving the algebra of observables. The key feature is that it is not unitary but pseudo-unitary, exactly in the same sense as the Minkowski spacetime representation. The language of pseudo-Hermitian quantum mechanics is adopted for a clear illustration of the aspect, with a metric operator obtained as really the manifestation of the Minkowski metric on the space of the state vectors. Explicit wavefunction description is given without any restriction of the variable domains, yet with a finite integral inner product. The associated covariant harmonic oscillator Fock state basis has all the standard properties in exact analog to those of a harmonic oscillator with Euclidean position and momentum operators. Galilean limit and the classical limit are retrieved rigorously through appropriate symmetry contractions of the algebra and its representation, including the dynamics described through the symmetry of the phase space.