Pathological conditions including cancers lead to accumulation of a morphological mixture of highly immunosuppressive cells termed as myeloid-derived suppressor cells (MDSC). The lack of conclusive markers to identify human MDSC, due to their heterogeneous nature and close phenotypical and functional proximity with other cell subsets, made it challenging to identify these cells. Nevertheless, expansion of MDSC has been reported in periphery and tumor microenvironment of various cancers. The majority of studies on breast cancers were performed on murine models and hence limited literature is available on the relation of MDSC accumulation with clinical settings in breast cancer patients. The aim of this study was to investigate levels and phenotypes of myeloid cells in peripheral blood (n = 23) and tumor microenvironment of primary breast cancer patients (n = 7), compared with blood from healthy donors (n = 21) and paired non-tumor normal breast tissues from the same patients (n = 7). Using multicolor flow cytometric assays, we found that breast cancer patients had significantly higher levels of tumor-infiltrating myeloid cells, which comprised of granulocytes (P = 0.022) and immature cells that lack the expression of markers for fully differentiated monocytes or granulocytes (P = 0.016). Importantly, this expansion was not reflected in the peripheral blood. The immunosuppressive potential of these cells was confirmed by expression of Arginase 1 (ARG1), which is pivotal for T-cell suppression. These findings are important for developing therapeutic modalities to target mechanisms employed by immunosuppressive cells that generate an immune-permissive environment for the progression of cancer.Electronic supplementary materialThe online version of this article (doi:10.1007/s00262-017-1977-z) contains supplementary material, which is available to authorized users.