Breast cancer (BC) has traditionally been considered to be not inherently immunogenic and insufficiently represented by immune cell infiltrates. Therefore, for a long time, it was thought that the immunotherapies targeting this type of cancer and its microenvironment were not justified and would not bring benefits for breast cancer patients. Nevertheless, to date, a considerable number of reports have indicated tumor-infiltrating lymphocytes (TILs) as a prognostic and clinically relevant biomarker in breast cancer. A high TILs expression has been demonstrated in primary tumors, of both, HER2-positive BC and triple-negative (TNBC), of patients before treatment, as well as after treatment with adjuvant and neoadjuvant chemotherapy. Another milestone was reached in advanced TNBC immunotherapy with the help of the immune checkpoint inhibitors directed against the PD-L1 molecule. Although those findings, together with the recent developments in chimeric antigen receptor T cell therapies, show immense promise for significant advancements in breast cancer treatments, there are still various obstacles to the optimal activity of immunotherapeutics in BC treatment. Of these, the immunosuppressive tumor microenvironment constitutes a key barrier that greatly hinders the success of immunotherapies in the most aggressive types of breast cancer, HER2-positive and TNBC. Therefore, the improvement of the current and the demand for the development of new immunotherapeutic strategies is strongly warranted.