Protopanaxadiol (PPD), a main ginseng metabolite, exerts powerful anticancer effects against multiple types of cancer; however, its cellular targets remain elusive. Here, we synthesized a cell-permeable PPD probe via introducing a bifunctional alkyne-containing diazirine photo-crosslinker and performed a photoaffinity labeling-based chemoproteomic study. We identified retinoblastoma binding protein 4 (RBBP4), a chromatin remodeling factor, as an essential cellular target of PPD in HCT116 colorectal cancer cells. PPD significantly decreased RBBP4-dependent trimethylation at lysine 27 of histone H3 (H3K27me3), a crucial epigenetic marker that correlates with histologic signs of colorectal cancer aggressiveness, and PPD inhibition of proliferation and migration of HCT116 cells was antagonized by RBBP4 RNA silencing. Collectively, our study highlights a previously undisclosed anti-colorectal cancer cellular target of the ginseng metabolite and advances the fundamental understanding of RBBP4 functions via a chemical biology strategy.