Developing high yielding cultivars with outstanding quality traits are perpetual objectives throughout crop breeding operations. Confoundingly, both of these breeding objectives typically involve working with complex quantitative traits that can be affected by genetic and environmental factors. Establishing correlations of these complex traits with more easily identifiable and highly heritable traits can simplify breeding processes. In this study, two parental soybean genotypes contrasting in seed hilum size, yield, and seed quality, as well as 175 F9 recombinant inbred lines (RILs) derived from these parents, were grown in 3 years. The h2b of four hilum size, two quality and two yield traits, ranged from 0.72 to 0.87. The four observed hilum size traits exhibited significant correlation (P < 0.05) with most of seed yield and quality traits, as indicated by correlation coefficients varying from -0.35 to 0.42, which suggests that hilum size could be considered as a proxy trait for soybean yield and quality. Interestingly, among 53 significant quantitative trait loci (QTLs) with logarithm of odds (LOD) values ranging from 2.51 to 6.69 and accounting for 6.40–16.10% of genetic variation, three loci encoding hilum size, qSH6.2, qSH8, and qSH10, colocated with QTLs for seed yield and quality traits, demonstrating that genes impacting seed hilum size colocalize in part with genes acting on soybean yield and quality. As a result of the breeding efforts and field observations described in this work, it is reasonable to conclude that optimizing hilum size through selection focused on a few QTLs may be useful for breeding new high yielding soybean varieties with favorable quality characteristics.