Background In recent years, it has been reported that diabetic patients tend to have a lower zinc intake due to unbalanced diet accompanying changes in lifestyle habits. We investigated serum zinc concentration in diabetic patients according to the stage of nephropathy. Methods We enrolled 227 diabetic patients (119 men, 108 women, average age 65.7 ± 14.7 [mean ± standard deviation]) who were hospitalized for diabetes treatment due to poor blood glucose control. We investigated the relationship between fasting serum zinc concentration and estimated glomerular filtration rate (eGFR) and albuminuria (urinary albumin-to-creatinine ratio, UACR), as well as serum zinc concentration by stage of diabetic kidney disease and chronic kidney disease. Results The mean HbA1c value was 10.5 ± 2.1%. Serum zinc concentration was 75.5 ± 16.0 μg/dL in males and 75.7 ± 12.2 μg/dL in females, showing no gender difference and no significant relationship with diabetes type. The serum zinc concentration was negatively correlated with age (r = − 0.309, P < 0.001) and positively correlated with eGFR (r = 0.144, P = 0.030). A tendency was observed of serum zinc concentration to decrease after overt nephropathy, with values of 76.4 ± 14.1 μg/dL in pre-nephropathy (stage 1, n = 131), 78.5 ± 13.2 μg/dL in incipient nephropathy (stage 2, n = 65), 66.4 ± 14.3 μg/dL in overt nephropathy (stage 3, n = 25), and 65.7 ± 11.9 μg/dL in kidney failure (stage 4, n = 6). Serum zinc showed a negative trend with estimated GFR (P = 0.004) and significant reduction in albuminuria, with stage A3 (n = 29, 65.7 ± 13.9 μg/dL) having lower levels than A1 (n = 131, 76.4 ± 14.1 μg/dL, P = 0.001) and A2 (n = 67, 78.4 ± 13.1 μg/dL, P < 0.001). Conclusions In diabetic patients, serum zinc concentration tended to decrease as age increased and also as renal function deteriorated. This study suggests that consideration of zinc deficiency is necessary in patients with overt albuminuria.