Kv3 channels have distinctive gating kinetics tailored for rapid repolarization in fast-spiking neurons. Malfunction of this process due to genetic variants in the KCNC1 gene causes severe epileptic disorders, yet the structural determinants for the unusual gating properties remain elusive. Here, we present cryo-EM structures of the human Kv3.1a channel, revealing a unique arrangement of the cytoplasmic T1 domain which facilitates interactions with C-terminal axonal targeting motif and key components of the gating machinery. Additional interactions between S1/S2 linker and turret domain strengthen the VSD/PD interface. Supported by MD simulations and electrophysiological and mutational analyses, we identify close communication between α6 helix of T1 domain, S4/S5 linker and S6T helix as responsible for the ultra-fast activation/deactivation and open state stabilisation that are unique to Kv3 channels. These findings provide fundamentally new insights into gating control and disease mechanisms and guide strategies for the design of pharmaceutical drugs targeting Kv3 channels.