As an essential metal, zinc is central to insulin biosynthesis and energy metabolism. Zinc can not only maintain the activity of insulin, but also has insulin-like effect. When zinc is sufficient, the body needs less insulin. Zinc can correct abnormal glucose tolerance and even replace insulin to improve glucose metabolism disorder in diabetic rats. However, the effect of paternal zinc deficiency on glucose metabolism of offspring is still unclear. In the present study, sixteen 8-week-old male mice were randomly allocated into low-zinc group and control group (8 mice in each group), which were fed with low zinc and standard diet for 6 weeks, respectively. The mice were mated with female mice fed with standard diet to get the first generation of mice (F1) to explore the effect zinc deficiency on the glucose metabolism of offspring. Glucose tolerance, insulin sensitivity and insulin secretion in mice were determined by oral glucose tolerance test (OGTT), insulin tolerance test (ITT) and glucose stimulated insulin secretion test (GSIS), respectively. Compared with the control group, the fasting blood glucose levels of F1 generation male mice in the low zinc group increased at 15 and 30 min after glucose injection. The blood glucose of F1 generation male mice in the low zinc group of mice decreased at 60, 90, 120, 180, 240 min after insulin injection. Compared with the control group, serum insulin of F1 generation male mice in the low zinc group decreased at 15 min after glucose injection (all p values <0.05). However F1 female mice in the low zinc group did not show abnormal glucose metabolism. Marginal zinc deficiency in male mice can cause abnormal glucose homeostasis in male offspring, but not in female offspring.