Immunotherapy exhibited potential effects for advanced hepatocellular carcinoma, unfortunately, the clinical benefits are often countered by cancer adaptive immune suppressive response. Uncovering the mechanism how cancer cells evade immune surveillance would help to develop new immunotherapy approaches and combination therapy. In this article, by analyzing the transcriptional factors which modulate the differentially expressed genes between T cell infiltration high group and low group, we identified oncoprotein B cell lymphoma 6 (BCL6) suppresses the infiltration and activation of tumor infiltrating T lymphocytes, thus correlated with poorer clinical outcome. By using antibody deletion experiment, we further demonstrated that CD4+T cells but not CD8+T cells are the main lymphocyte population suppressed by Bcl6 to promote HCC development. Mechanistically, BCL6 decreases cancer cell expression of pro-inflammatory cytokines and T lymphocyte chemokines such as IL6, IL1F6, and CCL5. Moreover, BCL6 upregulates Endothelial cell-specific molecule 1 (ESM1) to inhibit T lymphocyte recruitment and activation possibly through ICAM-1/LFA-1 signaling pathway. Our findings uncovered an unappreciated paracrine mechanism how cancer cell-derived BCL6 assists cancer cell immune evasion, and highlighted the role of CD4+T cells in HCC immune surveillance.