CoMn2O4 nanoparticles synthesized through facile co-precipitation technique, exhibit a mixed phase of tetragonal and cubic at room temperature. Rietveld fitting of the XRD pattern specifies 91.84% of the tetragonal phase and 8.16% of the cubic phase. Raman spectra and selected area electron diffraction pattern further confirm the structure. Transmission electron micrograph confirms the semi-spherical shape of the particles with average diameter 95 nm, are polycrystalline in nature. While 97% of Mn3+ and 3% of Co3+ occupy the octahedral site in the tetragonal phase, 52% of Mn3+ and 48% of Co2+ occupy the octahedral site in the cubic phase. XPS analysis further confirms the presence of both +2 and +3 oxidation states of Co and Mn. Magnetic measurement shows two magnetic transitions, Tc1 at 165 K and Tc2 at 93 K corresponding to paramagnetic to a lower magnetically ordered ferrimagnetic state followed by a higher magnetically ordered ferrimagnetic state, respectively. While Tc1 could be due to the cubic phase having inverse spinel structure, Tc2 is attributed to the tetragonal phase with normal spinel. The non-collinear triangular spin canting configuration of Mn3+ cations in octahedral sites while results in large magnetization near Tc2, the decrement in magnetization at the vicinity of Tc1 indicates the decrement in Mn3+ in octahedral site, well corroborated with XRD. In contrast to general temperature dependent HC observed in ferrimagnetic material, an unusual temperature dependent HC with high spontaneous exchange bias of 2.971 kOe and conventional exchange bias of 3.316 kOe at 50 K are observed. Interestingly, a high vertical magnetization shift (VMS) of 2.5 emu/g is observed at 5 K, attributed to the Yafet-Kittel spin structure of Mn3+ in the octahedral site. Such VMS has the potential to revolutionize the future of ultrahigh density magnetic recording technology.