Background: The impact of hepatitis E virus (HEV) infection on cancer development has been poorly investigated. This study aimed to explore the relationship between HEV seroprevalence and cancer risks and to identify high cancer risk subgroups in HEV-exposed populations. Methods: HEV seroprevalence status was determined in cancer and non-cancer subjects. Logistic regression and sensitivity analyses were used to assess the relationship between HEV antibody seropositivity and cancer risk for 17 cancer types. Additionally, interaction analyses were applied to interpret the association of HEV seroprevalence and other cancer risk factors. Results: Of the enrolled 4948 cancer and 4948 non-cancer subjects, cancer subjects had a higher anti-HEV seropositivity than non-cancer subjects (46.36% vs. 32.50%, p < 0.01). However, this divergency varied in degrees across different cancer types. Additionally, HEV seroprevalence was associated with cancer risk in young males (OR: 1.64, 95% CI: 1.19–2.27, p < 0.01). Remarkably, a significant association between HEV seroprevalence and cancer risk was observed only in gastric cancer patients (OR: 1.82, 95% CI: 1.07–3.09, p = 0.03). Conclusions: HEV seroprevalence was associated with cancer risk selectively in gastric cancer patients and young males, suggesting that cancer screening, particularly gastric cancer, should be regularly performed in young males with a history of HEV exposure.