The tumor bulk is composed of a highly heterogeneous population of cancer cells, as well as a large variety of resident and infiltrating host cells, extracellular matrix proteins, and secreted proteins, collectively known as the tumor microenvironment (TME). The TME is essential for driving tumor development by promoting cancer cell survival, migration, metastasis, chemoresistance, and the ability to evade the immune system responses. Therapeutically targeting tumor-associated macrophages (TAMs), cancer-associated fibroblasts (CAFs), regulatory T-cells (T-regs), and mesenchymal stromal/stem cells (MSCs) is likely to have an impact in cancer treatment. In this review, we focus on describing the normal physiological functions of each of these cell types and their behavior in the cancer setting. Relying on the specific surface markers and secreted molecules in this context, we review the potential targeting of these cells inducing their depletion, reprogramming, or differentiation, or inhibiting their pro-tumor functions or recruitment. Different approaches were developed for this targeting, namely, immunotherapies, vaccines, small interfering RNA, or small molecules.