Background/Objectives: Zinc is an essential microelement, and its deficiency is common in patients undergoing hemodialysis. However, the association between serum zinc and mortality in these patients remains unclear. The aim of this study was to explore the possible association between serum zinc levels and all-cause mortality in prevalent patients with kidney failure on maintenance hemodialysis. Methods: This was a prospective cohort study of maintenance hemodialysis patients followed up for 5 years. The key exposure was serum zinc level measured at baseline, and the outcome was all-cause mortality. Their association was analyzed using Cox proportional hazard models. Results: Among 1662 eligible patients selected for this analysis, 468 (28%) died. Lower serum zinc levels were associated with a higher risk for mortality, independent of the major demographic factors and factors including mineral and bone disorder and renal anemia. However, this association was no longer significant when adjusted for serum albumin. Because there was a close correlation between serum zinc and albumin levels, we performed further analyses in which participants were categorized into four groups by median serum zinc (68 µg/dL) and albumin (3.7 g/dL) levels. In the lower serum albumin groups, risk of death was significantly higher in those with lower zinc than those with higher zinc levels, whereas such a difference was not significant in the high serum albumin groups. Conclusions: In patients undergoing maintenance hemodialysis with lower serum albumin levels, a lower serum zinc level was associated with a higher risk of mortality.