Inducible T cell co‐stimulator (ICOS) is a positive immune checkpoint receptor expressed on the surface of activated T cells, which could promote cell function after being stimulated with ICOS ligand (ICOS‐L). Although clinical benefits have been reported in the ICOS modulation‐based treatment for cancer and autoimmune disease, current modulators are restricted in biologics, whereas ICOS‐targeted small molecules are lacking. To fill this gap, we performed an affinity selection mass spectrometry (ASMS) screening for ICOS binding using a library of 15,600 molecules. To the best of our knowledge, this is the first study that utilizes ASMS screening to discover small molecules targeting immune checkpoints. Compound 9 with a promising ICOS/ICOS‐L inhibitory profile (IC50 = 29.38 ± 3.41 µM) was selected as the template for the modification. Following preliminary structure‐activity relationship (SAR) study and molecular dynamic (MD) simulation revealed the critical role of the ortho‐hydroxy group on compound 9 in the ICOS binding, as it could stabilize the interaction via the hydrogen bond formation with residuals on the glycan, and the depletion could lead to an activity lost. This work validates a promising inhibitor for the ICOS/ICOS‐L interaction, and we anticipate future modifications could provide more potent modulators for this interaction.