The tumor microenvironment (TME) interacting with the malignant cells plays a vital role in cancer development. Herein, we aim to establish and verify a scoring system based on the characteristics of TME cells for prognosis prediction and personalized treatment guidance in patients with triple-negative breast cancer (TNBC). 158 TNBC samples from The Cancer Genome Atlas (TCGA) were included as the training cohort, and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) (N = 297), as well as GSE58812 (N = 107), were included as the validation cohort. The enrichment scores of 64 immune and stromal cells were estimated by the xCell algorithm. In the training cohort, cells with prognostic significance were found out using univariate Cox regression analysis and further applied to the random survival forest (RSF) model. Basing on the scores of M2 macrophages, CD8+ T cells, and CD4+ memory T cells, a risk scoring system was constructed, which divided TNBC patients into 4 phenotypes (M2low, M2highCD8+ThighCD4+Thigh, M2highCD8+ThighCD4+Tlow, and M2highCD8+Tlow) and 2 groups. The low-risk group had superior survival outcomes than the high-risk one, which was further confirmed in the validation cohort. Moreover, in the low-risk group, immune-related pathways were significantly enriched, and a higher level of antitumoral immune cells and immune checkpoint molecules, including PD-L1, PD-1, and CTLA-4, could be observed. Additionally, consistent results were achieved in the SYSUCC cohort when the scoring system was applied. In summary, this novel scoring system might predict the survival and immune activity of patients and might serve as a potential index for immunotherapy.