In recent decades, the increasing interest in the field of immunotherapy has fostered an intense investigation of the breast cancer (BC) immune microenvironment. In this context, tumor-infiltrating lymphocytes (TILs) have emerged as a clinically relevant and highly reproducible biomarker capable of affecting BC prognosis and response to treatment. Indeed, the evaluation of TILs on primary tumors proved to be strongly prognostic in triple-negative (TN) BC patients treated with either adjuvant or neoadjuvant chemotherapy, as well as in early TNBC patients not receiving any systemic treatment, thus gaining level-1b evidence in this setting. In addition, a strong relationship between TILs and pathologic complete response after neoadjuvant chemotherapy has been reported in all BC subtypes and the prognostic role of higher TILs in early HER2-positive breast cancer patients has also been demonstrated. The interest in BC immune infiltrates has been further fueled by the introduction of the first immune checkpoint inhibitors in the treatment armamentarium of advanced TNBC in patients with PD-L1-positive status by FDA-approved assays. However, despite these advances, a biomarker capable of reliably and exhaustively predicting immunotherapy benefit in BC is still lacking, highlighting the imperative need to further deepen this issue. Finally, more comprehensive evaluation of immune infiltrates integrating both the quantity and quality of tumor-infiltrating immune cells and incorporation of TILs in composite scores encompassing other clinically or biologically relevant biomarkers, as well as the adoption of software-based and/or machine learning platforms for a more comprehensive characterization of BC immune infiltrates, are emerging as promising strategies potentially capable of optimizing patient selection and stratification in the research field. In the present review, we summarize available evidence and recent updates on immune infiltrates in BC, focusing on current clinical applications, potential clinical implications and major unresolved issues.