The Endogenous Peptide Inhibitor of CXCR4 (EPI-X4) is a body-own fragment of albumin and specific antagonist of the CXC-motif-chemokine receptor 4 (CXCR4). CXCR4 signaling is induced by its sole chemokine ligand CXCL12 and is involved in a plethora of functions including cell homing, differentiation, survival and angiogenesis. Consequently, dysregulation of CXCR4 is involved in a variety of disorders, such as cancer or inflammatory diseases, making CXCR4 an attractive drug target. EPI-X4 and derivatives with increased CXCR4 binding affinities represent promising leads as CXCR4 antagonists and have shown therapeutic activity in mouse models of inflammatory diseases. However, it is currently unclear how EPI-X4 and its derivatives interact with CXCR4. Here, by combining biomolecular simulations with experimental mutagenesis and activity studies we investigated the binding behavior of EPI-X4 to CXCR4 at the molecular level. Our work allowed us to show that the EPI-X4 peptide interacts primarily in the minor pocket of CXCR4 through its N-terminal residues. The biomolecular interactions highlighted by the computational studies are in good agreement with the experimental mutagenesis data. Moreover, we found that the N-terminal seven amino-acids of EPI-X4 (a 16-mer) and its improved derivatives (12-mers) are sufficient for CXCR4 binding, which led to the development of shorter leads with optimized CXCR4 antagonizing properties. Collectively, we here established how EPI-X4 binds to its receptor and used this knowledge for rational drug design. The new peptide variants developed by us are more potent in terms of inhibiting CXCR4-downstream signaling and cancer cell migration, without toxic effects.