As the main cause of death in the world, cancer is one of the major health threats for humans. In recent years, traditional Chinese medicine has gained great attention in oncology due to the features of multi-targets, multi-pathways, and slight side effects. Moreover, lots of traditional Chinese medicine can exert immunomodulatory effects in vivo. In the tumor microenvironment, tumor cells, immune cells as well as other stromal cells often coexist. With the development of cancer, tumor cells proliferate uncontrollably, metastasize aggressively, and modulate the proportion and status of immune cells to debilitate the antitumor immunity. Reversal of immunosuppressive tumor microenvironment plays an essential role in cancer prevention and therapy. Immunotherapy has become the most promising strategy for cancer therapy. Chinese medicine compounds can stimulate the activation and function of immune cells, such as promoting the maturation of dendritic cells and inducing the differentiation of myeloid-derived suppressor cells to dendritic cells and macrophages. In the present review, we summarize and discuss the effects of Chinese medicine compounds on immune cells in the tumor microenvironment, including innate immune cells (dendritic cells, natural killer cells, macrophages, and myeloid-derived suppressor cells) and adaptive immune cells (CD4+/CD8+ T lymphocytes and regulatory T cells), and the various immunomodulatory roles of Chinese medicine compounds in cancer therapy such as improving tumor-derived inflammation, enhancing the immunity after surgery or chemotherapy, blocking the immune checkpoints, et al., aiming to provide more thoughts for the anti-tumor mechanisms and applications of Chinese medicine compounds in terms of tumor immunity.