In maize hybrid seed production, some hybrid seed in the field must be harvested before reaching physiological maturity because of the potential damage from early fall frosts. However, early harvesting can result in poor quality and low vigor of seeds. To elucidate the genetic basis of seed vigor at different stages of maturity, the seeds of a set of recombinant inbred line (RIL) populations at three different stages of maturity (32, 40, and 45 days after pollination; DAP), were used to evaluate the performance of four traits for seed vigor in the field. A genetic linkage map was constructed using 217 SSR makers covering 2438.2 cM with an average interval of 11.2 cM. The results showed that there were significant positive relationships among the four traits of seed vigor at all three sampling times, and all showed quantitative changes according to the degree of maturity of the seeds. However, the four traits of seed vigor were not significantly related to the 100-kernel weight. In total, we detected 16 different QTL for the four measured traits of seed vigor at three sampling times; five QTL were for germination energy, three for germination percentage, four for germination index, and four for vigor index. Interestingly, four QTL for seed vigor, which were detected at all three sampling times, were located in the same region on chromosome 7. This result implies that this region of chromosome 7 is important for seed vigor of seeds harvested before they reach physiological maturity.