Conception in dairy cattle is influenced by the fertility of the cow and the bull and their interaction. Despite genetic selection for female fertility in many countries, selection for male fertility is largely not practiced. The primary objective of this study was to quantify variation in male and female fertility using insemination data from predominantly seasonal-calving herds. Nonreturn rate (NRR) was derived by coding each insemination as successful (1) or failed (0) based on a minimum of at least 25 d. The NRR was treated as a trait of the bull with semen (male fertility) and the cow that is mated (female fertility). The data (805,463 cows that mated to 5,776 bulls) were used to estimate parameters using either models that only included bulls with mating data or models that fitted the genetic and permanent environmental (PE) effects of bulls and cows simultaneously. We also evaluated whether fitting genetic and PE effects of bulls as one term is better for ranking bulls based on NRR compared with a model that ignored genetic effect. The age of cows that were mated, age of the bulls with semen data, season of mating, breed of cow that mated, inbreeding of cows and bulls, and days from calving to mating date were found to have a significant effect on NRR. Only about 3% of the total variance was explained by the random effects in the model, despite fitting the genetic and PE effects of the bull and cow. The 2 components of fertility (male and fertility) were not correlated. The heritability of male fertility was low (0.001 to 0.008), and that of female fertility was also low (~0.016). The highest heritability estimate for male fertility was obtained from the model that fitted the additive genetic relationship matrix and PE component of the bull as one term. When this model was used to calculate bull solutions, the differ-ence between bulls with at least 100 inseminations was up to 19.2% units (−9.6 to 9.6%). Bull solutions from this model were compared with bull solutions that were predicted fitting bull effects ignoring pedigree. Bull solutions that were obtained considering pedigree had (1) the highest accuracy of prediction when early insemination was used to predict yet-to-be observed insemination data of bulls, and (2) improved model stability (i.e., a higher correlation between bull solutions from 2 randomly split herds) compared with the model which fitted bull with no pedigree. For practical purposes, the model that fitted genetic and PE effect as one term can provide more accurate semen fertility values for bulls than the model without genetic effect. To conclude, insemination data from predominantly seasonal-calving herds can be used to quantify variability between bulls for male fertility, which makes their ranking on NRR feasible. Potentially this information can be used for monitoring bulls and can supplement efforts to improve herd fertility by avoiding or minimizing the use of semen from subfertile bulls.