Adipose-derived mesenchymal stem cells (ASCs) are known to have immunomodulatory properties through soluble factors or by direct cell-to-cell contact. This study aimed to assess the expression of HLA-G and IDO activity in breast cancer and normal ASCs and to see whether ASC is capable of modulating both tumor cells and immune system cells in vitro. ASCs were enzymatically isolated from 15 breast cancer patients and 10 normal individuals. Then they were cultured, and the impact of their conditioned media on the movement of the MDA-MB-231 breast cancer cell line was studied in wound healing scratch assay. Next, PBLs from the peripheral blood of normal individuals were separated and co-cultured with breast cancer and normal ASCs. PBLs proliferation and apoptosis were assessed using CFSE labeling dye and annexin V/7AAD staining, respectively. IDO activity and HLA-G protein expression in ASCs were examined using kynurenine assay and Western blotting, respectively. Tumor-derived ASCs, especially those from higher stages of breast cancer, have stronger effects on the proliferation and movement of MDA-MB-231 cells than normal ASCs (P-value < 0.05). Apoptosis in PBLs increased in the presence of ASCs compared to PBLs cultured alone (P-value < 0.05). In contrast, necrosis of PBLs decreased in the presence of ASCs compared to apoptosis in these cells (P-value < 0.001). Collectively, ASCs may have strategic effects on both tumor cells and cells of the immune system in the tumor microenvironment, resulting in tumor development, growth, and metastasis.