Covariance matrices of spatially-correlated wireless channels in millimeter wave (mmWave) vehicular networks can be employed to design environment-aware beamforming codebooks. Such covariance matrices can be represented over non-Euclidean (i.e., curved surfaces) manifolds, thanks to their symmetric positive definite (SPD) structures. In this paper, we propose three learning models for channel covariance estimation over Riemannian manifolds. First, we propose an unsupervised Riemannian K-means clustering approach, where Log-Euclidean metric (LEM) is utilized as a distance metric among channel covariance matrices over the Riemannian manifold. Second, we propose a Riemannian Competitive Learning (RCL) model, which is an online clustering solution that is intended to reduce the learning time of the offline K-means models. Third, we apply a Riemannian Dictionary Learning (RDL) model that leverages the sparsity properties of mmWave channels, while estimating their channel covariance matrices. Simulation results show that the proposed Riemannian K-means, online RCL and RDL models can achieve 41%, 30% and 44% higher data rate, respectively, than their Euclidean-based counterparts. Furthermore, they require few training samples and hence fast construction of codebook design in dynamic environments, which leads to low latency communication.