Motivated by the proposal of topological quantum paramagnet in the diamond lattice antiferromagnet NiRh2O4, we propose a minimal model to describe the magnetic interaction and properties of the diamond material with the spin-one local moments. Our model includes the first and second neighbor Heisenberg interactions as well as a local single-ion spin anisotropy that is allowed by the spin-one nature of the local moment and the tetragonal symmetry of the system. We point out that there exists a quantum phase transition from a trivial quantum paramagnet when the single-ion spin anisotropy is dominant to the magnetic ordered states when the exchange is dominant. Due to the frustrated spin interaction, the magnetic excitation in the quantum paramagnetic state supports extensively degenerate band minima in the spectra. As the system approaches the transition, extensively degenerate bosonic modes become critical at the criticality, giving rise to unusual magnetic properties. Our phase diagram and experimental predictions for different phases provide a guildline for the identification of the ground state for NiRh2O4. Although our results are fundamentally different from the proposal of topological quantum paramagnet, it represents interesting possibilities for spin-one diamond lattice antiferromagnets.