The CXCR4/CXCL12 axis plays prominent roles in tumor metastasis and inflammation. CXCR4 has been shown to be involved in a variety of inflammation-related diseases. Therefore, CXCR4 is a promising potential target to develop novel anti-inflammatory agents. Taking our previously discovered CXCR4 modulator RB-108 as the lead compound, a series of derivatives were synthesized structurally modifying and optimizing the amide and sulfamide side chains. The derivatives successfully maintained potent CXCR4 binding affinity. Furthermore, compounds Ilb, IIc, IIIg, IIIj, and IIIm were all efficacious in inhibiting the invasion of CXCR4-positive cells, displaying a much more potent effect than the lead compound RB-108. Notably, compound IIIm significantly decreased carrageenan-induced swollen volume and paw thickness in a mouse paw edema model. More importantly, IIIm exhibited satisfying PK profiles with a half-life of 4.77 h in an SD rat model. In summary, we have developed compound IIIm as a new candidate for further investigation based on the lead compound RB-108.