We perform ultrasound velocity measurements on a single crystal of nearly-metallic spinel Co1.21V1.79O4 which exhibits a ferrimagnetic phase transition at TC ∼ 165 K. The experiments reveal a variety of elastic anomalies in not only the paramagnetic phase above TC but also the ferrimagnetic phase below TC, which should be driven by the nearly-itinerant character of the orbitally-degenerate V 3d electrons. In the paramagnetic phase above TC , the elastic moduli exhibit elastic-mode-dependent unusual temperature variations, suggesting the existence of a dynamic spin-cluster state. Furthermore, above TC, the sensitive magnetic-field response of the elastic moduli suggests that, with the negative magnetoresistance, the magnetic-field-enhanced nearly-itinerant character of the V 3d electrons emerges from the spin-cluster state. This should be triggered by the inter-V-site interactions acting on the orbitally-degenerate 3d electrons. In the ferrimagnetic phase below TC , the elastic moduli exhibit distinct anomalies at T1 ∼ 95 K and T2 ∼ 50 K, with a sign change of the magnetoresistance at T1 (positive below T1) and an enhancement of the positive magnetoresistance below T2, respectively. These observations below TC suggest the successive occurrence of an orbital glassy order at T1 and a structural phase transition at T2, where the rather localized character of the V 3d electrons evolves below T1 and is further enhanced below T2.