Tissue-resident memory T (TRM) cells mediate potent local innate and adaptive immune responses and provide long-lasting protective immunity. TRM cells localize to many different tissues, including barrier tissues, and play a crucial role in protection against infectious and malignant disease. The formation and maintenance of TRM cells are influenced by numerous factors, including inflammation, antigen triggering, and tissue-specific cues. Emerging evidence suggests that these signals also contribute to heterogeneity within the TRM cell compartment. Here, we review the phenotypic and functional heterogeneity of CD8+ TRM cells at different tissue sites and the molecular determinants defining CD8+ TRM cell subsets. We further discuss the possibilities of targeting the unique cell surface molecules, cytokine and chemokine receptors, transcription factors, and metabolic features of TRM cells for therapeutic purposes. Their crucial role in immune protection and their location at the frontlines of the immune defense make TRM cells attractive therapeutic targets. A better understanding of the possibilities to selectively modulate TRM cell populations may thus improve vaccination and immunotherapeutic strategies employing these potent immune cells.