Gene regulation is complex, involving the coordination of hundreds of proteins that function to control genome accessibility, mediate enhancer-promoter interactions, and initiate transcription. At individual loci, transcriptional initiation is stochastic, resulting in short periods of nascent RNA synthesis known as transcriptional bursts. To understand how altered Estrogen Receptor function and cofactor recruitment regulates transcriptional bursting, we used single molecule imaging of estrogen responsive genes in Bisphenol A (BPA) treated cells. Using live cell imaging of the estrogen responsive TFF1 gene, we observe that cells treated with BPA exhibited burst initiation kinetics and burst sizes which were indistinguishable from cells induced with Estradiol (E2). However, we observed a 50% reduction in the number of active alleles in BPA treated cells. This effect is gene specific, as GREB1 was unperturbed. Although we observed no difference in chromatin accessibility, the TFF1 promoter exhibited an altered structure which coincided with reduced ERα and cofactor binding. Lastly, deletion of the enhancer locus removed the BPA effect, indicating that enhancer function was perturbed. Our results demonstrate gene specific effects of altered ERα recruitment and function which lead to a reduction of transcriptionally permissive states. Our work supports the model that the early estrogen response occurs from alleles in primed transcriptionally permissive states with additional inactive alleles contributing to the response over time.