The vectorial (5′-to-3′ at varying velocity) synthesis of RNA by cellular RNA polymerases creates a rugged kinetic landscape, demarcated by frequent, sometimes long-lived pauses. In addition to myriad gene-regulatory roles, these pauses temporally and spatially program the co-transcriptional, hierarchical folding of biologically active RNAs. Conversely, these RNA structures, which form inside or near the RNA exit channel, interact with the polymerase and adjacent protein factors to influence RNA synthesis by modulating pausing, termination, antitermination, and slippage. Here we review the evolutionary origin, mechanistic underpinnings, and regulatory consequences of this interplay between RNA polymerase and nascent RNA structure. We categorize and attempt to rationalize the extensive linkage between the transcriptional machinery and its product, and provide a framework for future studies.