Temporal lobe epilepsy (TLE) is one of the syndromes linked to antibodies against glutamic acid decarboxylase (GAD). It has been questioned whether ‘limbic encephalitis with GAD antibodies’ is a meaningful diagnostic entity. The immunopathogenesis of GAD-TLE has remained enigmatic. Improvement of immunological treatability is an urgent clinical concern. We retrospectively assessed the clinical, MRI and CSF course as well as brain tissue of 15 adult patients with GAD-TLE who underwent temporal lobe surgery. Brain tissue was studied by means of immunohistochemistry, multiplex fluorescent microscopy and transcriptomic analysis for inflammatory mediators and neuronal degeneration. In 10 patients, there was a period of mediotemporal swelling and T2 signal increase; in nine cases this occurred within the first 6 years after symptom onset. This resulted in unilateral or bilateral hippocampal sclerosis (HS); three cases developed HS already within the first 2 years. All CSF studies done within the first year (n = 6) revealed intrathecal synthesis of immunoglobulin G. Temporal lobe surgeries were done after a median disease duration of 9 years (range 3 weeks to 60 years). Only two patients became seizure free. Brain parenchyma collected during surgery in the first 6 years revealed high numbers of plasma cells but no signs of antibody-mediated tissue damage. Even more dense was the infiltration by CD8 + cytotoxic T lymphocytes (CTLs) that were seen to locally proliferate. Further, a portion of these cells revealed an antigen-specific resident memory T cell phenotype. Finally, CTLs with cytotoxic granzyme B+ granules were also seen in microglial nodules and attached to neurons, suggesting a CTL-mediated destruction of these cells. With longer disease duration, the density of all lymphocytes decreased. Whole transcriptome analysis in early/active cases (but not in late/inactive stages) revealed ‘T cell immunity’ and ‘Regulation of immune processes’ as the largest overrepresented clusters. To a lesser extent, pathways associated with B cells and neuronal degeneration also showed increased representation. Surgically treated patients with GAD-TLE go through an early active inflammatory, ‘encephalitic’ stage (≤ 6 years) with CTL-mediated, antigen-driven neuronal loss and antibody-producing plasma cells but without signs of complement-mediated cell death. Subsequently, patients enter an apparently immunologically inactive or low-active stage with ongoing seizures, probably caused by the structural damage to the temporal lobe. ‘Limbic encephalitis’ with GAD antibodies should be subsumed under GAD-TLE. The early tissue-damage explains why immunotherapy does usually not lead to seizure-freedom.