The COVID-19 pandemic represents an ongoing global crisis that has already impacted over 13 million people. The responses of specific immune cell populations to the disease remain poorly defined, which hinders improvements in treatment and care management. Here, we utilized mass cytometry (CyTOF) to thoroughly phenotype peripheral myeloid cells and T lymphocytes from 30 convalescent patients with mild, moderate, and severe cases of COVID-19. We identified 10 clusters of monocytes and dendritic cells and 17 clusters of T cells. Examination of these clusters revealed that both CD14+CD16+ intermediate and CD14dimCD16+ nonclassical monocytes, as well as CD4+ stem cell memory T (TSCM) cells, correlated with COVID-19 severity, coagulation factor levels, and/or inflammatory indicators. We also identified two nonclassical monocyte subsets distinguished by expression of the sugar residue 6-Sulfo LacNac (Slan). One of these subsets (Slanlo, nMo1) was depleted in moderately and severely ill patients, while the other (Slanhi, nMo2) increased with disease severity and was linked to CD4+ T effector memory (TEM) cell frequencies, coagulation factors, and inflammatory indicators. Intermediate monocytes tightly correlated with loss of naive T cells as well as an increased abundance of effector memory T cells expressing the exhaustion marker PD-1. Our data suggest that both intermediate and non-classical monocyte subsets shape the adaptive immune response to SARS-CoV-2. In summary, our study provides both broad and in-depth characterization of immune cell phenotypes in response to COVID-19 and suggests functional interactions between distinct cell types during the disease.