The transcription factor glucocorticoid receptor (GR) is a key mediator of stress response and a broad range of physiological processes. How can GR rapidly activate the expression of some genes while repress others, remains an open question due to the challenge to associate GR binding sites (GBSs) to their distant gene targets. Mapping the full 3D scope of GR-responsive promoters using highresolution 4C-seq unravelled spatial separation between chromatin interaction networks of GRactivated and repressed genes. Analysing GR binding sites and other regulatory loci in their functional 3D context revealed that GR sequesters the co-activator Ep300 from active non-GBS enhancers in both activated and repressed gene compartments. While this is sufficient for rapid gene repression, gene activation is countered by productive recruitment of Ep300 to GBS. Importantly, in GR-activated compartments Klf4 binding at non-GBS regulatory elements cluster in 3D with GBS and antagonizes GR activation. In addition, we revealed ROR and Rev-erb transcription factors as novel co-regulators for GR-mediated gene expression.