In the present study, we evaluated four small molecule affinity-based probes based on agarose-immobilized benzamidine (ABA), O-Phospho-L-Tyrosine (pTYR), 8-Amino-hexyl-cAMP (cAMP), or 8-Amino-hexyl-ATP (ATP) for their ability to remove high-abundant proteins such as serum albumin from plasma samples thereby enabling the detection of medium-to-low abundant proteins in plasma samples by mass spectrometry-based proteomics. We compared their performance with the most commonly used immunodepletion method, the Multi Affinity Removal System Human 14 (MARS14) targeting the top 14 most abundant plasma proteins and also the ProteoMiner protein equalization method by label-free quantitative liquid chromatography tandem mass spectrometry (LC-MSMS) analysis. The affinity-based probes demonstrated a high reproducibility for low-abundant plasma proteins, down to picomol per mL levels, compared to the Multi Affinity Removal System (MARS) 14 and the Proteominer methods, and also demonstrated superior removal of the majority of the high-abundant plasma proteins. The ABA-based affinity probe and the Proteominer protein equalization method performed better compared to all other methods in terms of the number of analyzed proteins. All the tested methods were highly reproducible for both high-abundant plasma proteins and low-abundant proteins as measured by correlation analyses of six replicate experiments. In conclusion, our results demonstrated that small-molecule based affinity-based probes are excellent alternatives to the commonly used immune-depletion methods for proteomic biomarker discovery studies in plasma. Data are available via ProteomeXchange with identifier PXD020727.