Recently, small-molecule compounds have been reported to block the PD-1/PD-L1 interaction by inducing the dimerization of PD-L1. All these inhibitors had a common scaffold and interacted with the cavity formed by two PD-L1 monomers. This special interactive mode provided clues for the structure-based drug design, however, also showed limitations for the discovery of small-molecule inhibitors with new scaffolds. In this study, we revealed the structure-activity relationship of the current small-molecule inhibitors targeting dimerization of PD-L1 by predicting their binding and unbinding mechanism via conventional molecular dynamics and metadynamics simulation. During the binding process, the representative inhibitors (BMS-8 and BMS-1166) tended to have a more stable binding mode with one PD-L1 monomer than the other and the small-molecule inducing PD-L1 dimerization was further stabilized by the non-polar interaction of Ile54, Tyr56, Met115, Ala121, and Tyr123 on both monomers and the water bridges involved in ALys124. The unbinding process prediction showed that the PD-L1 dimerization kept stable upon the dissociation of ligands. It's indicated that the formation and stability of the small-molecule inducing PD-L1 dimerization was the key factor for the inhibitory activities of these ligands. The contact analysis, R-group based quantitative structure-activity relationship (QSAR) analysis and molecular docking further suggested that each attachment point on the core scaffold of ligands had a specific preference for pharmacophore elements when improving the inhibitory activities by structural modifications. Taken together, the results in this study could guide the structural optimization and the further discovery of novel small-molecule inhibitors targeting PD-L1.