A diagonalizable non-Hermitian Hamiltonian having a real spectrum may be used to define a unitary quantum system, if one modifies the inner product of the Hilbert space properly. We give a comprehensive and essentially self-contained review of the basic ideas and techniques responsible for the recent developments in this subject. We provide a critical assessment of the role of the geometry of the Hilbert space in conventional quantum mechanics to reveal the basic physical principle motivating our study. We then offer a survey of the necessary mathematical tools, present their utility in establishing a lucid and precise formulation of a unitary quantum theory based on a non-Hermitian Hamiltonian, and elaborate on a number of relevant issues of fundamental importance. In particular, we discuss the role of the antilinear symmetries such as PT , the true meaning and significance of the so-called charge operators C and the CPT -inner products, the nature of the physical observables, the equivalent description of such models using ordinary Hermitian quantum mechanics, the pertaining duality between localnon-Hermitian versus nonlocal-Hermitian descriptions of their dynamics, the corresponding classical systems, the pseudo-Hermitian canonical quantization scheme, various methods of calculating the (pseudo-) metric operators, subtleties of dealing with time-dependent quasi-Hermitian Hamiltonians and the path-integral formulation of the theory, and the structure of the state space and its ramifications for the quantum Brachistochrone problem. We also explore some concrete physical applications and manifestations of the abstract concepts and tools that have been developed in the course of this investigation. These include applications in nuclear physics, condensed matter physics, relativistic quantum mechanics and quantum field theory, quantum cosmology, electromagnetic wave propagation, open quantum systems, magnetohydrodynamics, quantum chaos, and biophysics.