Background: T-cell responses can be protective or detrimental during severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection; however, the underlying mechanism is poorly understood. Methods: In this study, we screened 144 15-mer peptides spanning the SARS-CoV-2 spike, nucleocapsid (NP), M, ORF8, ORF10, and ORF3a proteins and 39 reported SARS-CoV-1 peptides in peripheral blood mononuclear cells (PBMCs) from nine laboratory-confirmed coronavirus disease 2019 (COVID-19) patients (five moderate and four severe cases) and nine healthy donors (HDs) collected before the COVID-19 pandemic. T-cell responses were monitored by IFN-γ and IL-17A production using ELISA, and the positive samples were sequenced for the T cell receptor (TCR) β chain. The positive T-cell responses to individual SARS-CoV-2 peptides were validated by flow cytometry. Results: COVID-19 patients with moderate disease produced more IFN-γ than HDs and patients with severe disease (moderate vs. HDs, p < 0.0001; moderate vs. severe, p < 0.0001) but less IL-17A than those with severe disease (p < 0.0001). A positive correlation was observed between IFN-γ production and T-cell clonal expansion in patients with moderate COVID-19 (r = 0.3370, p = 0.0214) but not in those with severe COVID-19 (r = −0.1700, p = 0.2480). Using flow cytometry, we identified that a conserved peptide of the M protein (Peptide-120, P120) was a dominant epitope recognized by CD8+ T cells in patients with moderate disease. Conclusion: Coordinated IFN-γ production and clonal expansion of SARS-CoV-2-specific T cells are associated with disease resolution in COVID-19. Our findings contribute to a better understanding of T-cell-mediated immunity in COVID-19 and may inform future strategies for managing and preventing severe outcomes of SARS-CoV-2 infection.