The tumor microenvironment (TME) plays a key role in promoting the initiation and progression of tumors, leading to chemoradiotherapy resistance and immunotherapy failure. Targeting of the TME is a novel anti-tumor therapeutic approach and is currently a focus of anti-tumor research. Panax ginseng C. A. Meyer (ginseng), an ingredient of well-known traditional Asia medicines, exerts beneficial anti-tumor effects and can regulate the TME. Here, we present a systematic review that describes the current status of research efforts to elucidate the functions and mechanisms of ginseng active components (including ginsenosides and ginseng polysaccharides) for achieving TME regulation. Ginsenosides have variety effects on TME, such as Rg3, Rd and Rk3 can inhibit tumor angiogenesis; Rg3, Rh2 and M4 can regulate the function of immune cells; Rg3, Rd and Rg5 can restrain the stemness of cancer stem cells. Ginseng polysaccharides (such as red ginseng acidic polysaccharides and polysaccharides extracted from ginseng berry and ginseng leaves) can regulate TME mainly by stimulating immune cells. In addition, we propose a potential mechanistic link between ginseng-associated restoration of gut microbiota and the tumor immune microenvironment. Finally, we describe recent advances for improving ginseng efficacy, including the development of a nano-drug delivery system. Taken together, this review provides novel perspectives on potential applications for ginseng active ingredients as anti-cancer adjuvants that achieve anti-cancer effects by reshaping the tumor microenvironment.