The water buffalo (Bubalus bubalis) is an indispensable part of the Indian dairy sector and in several instances, the farmers incur economic losses due to failed pregnancy after artificial insemination (AI). One of the key factors for the failure of conception is the use of semen from the bulls of low fertilizing potential and hence, it becomes important to predict the fertility status before performing AI. In this study, the global proteomic profile of high fertile (HF) and low fertile (LF) buffalo bull spermatozoa was established using a high-throughput LC-MS/MS technique. A total of 1,385 proteins (≥1 high-quality PSM/s, ≥1 unique peptides, p < 0.05, FDR < 0.01) were identified out of which, 1,002 were common between both the HF and LF groups while 288 and 95 proteins were unique to HF and LF groups respectively. We observed 211 and 342 proteins were significantly high (log Fc ≥ 2) and low abundant (log Fc ≤ 0.5) in HF spermatozoa (p < 0.05). Gene ontology analysis revealed that the fertility associated high abundant proteins in HF were involved in spermatogenesis, sperm motility, acrosome integrity, zona pellucida binding and other associated sperm functions. Besides this, the low abundant proteins in HF were involved in glycolysis, fatty acid degradation and inflammation. Furthermore, fertility related differentially abundant proteins (DAPs) on sperm viz., AKAP3, Sp17, and DLD were validated through Western blotting and immunocytochemistry which was in coherence with the LC-MS/MS data. The DAPs identified in this study may be used as potential protein candidates for predicting fertility in buffaloes. Our findings provide an opportunity in mitigating the economic losses that farmers incur due to male infertility.