Colorectal cancer (CRC) is the third most common cancer worldwide, ranking as high as the second leading cause of cancer-related deaths in industrialized countries. Consistent with immunosurveillance theory, the immune system is crucial to protect the host from developing tumors, and the major players in tumoral immunity are effector T cells. Anyway, cancer cells develop strategies of immunoevasion influencing the cancer-specific lymphocyte priming, activation, and effector function. Therefore, the T cell subsets that mature during the stages of tumor growth, differently contribute to disease progression and/or regression. In our study, we analyzed the intra-tumoral and peripheral T cell subsets’ distribution in 30 patients with CRC, in order to clarify their functional role toward cancer. We found that percentage of infiltrating effector T cells decreased in cancer tissue than in healthy mucosa and that the tumor microenvironment negatively influences the cytolytic activity of T lymphocytes reactive to cancer cells. Moreover, we found that the tumor tissue was infiltrated by a large amount of “not effector” T (neT) cells with a regulatory or an anergic profile, which are unable to kill cancer cells, may be contributing to the CRC promotion. The presence of neT cells was investigated also in the peripheral blood of CRC patients, demonstrating that the peripheral T regulatory cells can inhibit the proliferation of effector T cells, confirming their immunosuppressive properties. Finally, monitoring the changes in circulating neT cells’ frequencies after the tumor removal, we confirmed the role of cancer in the modulation of immune system, in particular, in supporting a Tregs-mediated immunosuppression.