The interplay between cancer cells and the tumor microenvironment (TME) has a fundamental role in tumor progression and response to therapy. The plethora of components constituting the TME, such as stroma, fibroblasts, endothelial and immune cells, as well as macromolecules, e.g., hormones and cytokines, and epigenetic factors, such as microRNAs, can modulate the survival or death of cancer cells. Actually, the TME can stimulate the genetically regulated programs that the cell puts in place under stress: apoptosis or, of interest here, autophagy. However, the implication of autophagy in tumor growth appears still undefined. Autophagy mainly represents a cyto-protective mechanism that allows cell survival but, in certain circumstances, also leads to the blocking of cell cycle progression, possibly leading to cell death. Since significant sex/gender differences in the incidence, progression and response to cancer therapy have been widely described in the literature, in this review, we analyzed the roles played by key components of the TME, e.g., estrogen and microRNAs, on autophagy regulation from a sex/gender-based perspective. We focused our attention on four paradigmatic and different forms of cancers—colon cancer, melanoma, lymphoma, and lung cancer—concluding that sex-specific differences may exert a significant impact on TME/cancer interaction and, thus, tumor growth.