Oilseed rape is one of the world’s largest oil and industrial crops, providing humans with various products, such as vegetable oil and biofuel. Ovules are the direct precursors of seeds, and ovule number per ovary (ONPO) largely determines seed number per fruit that affects both yield and fitness of seed crops. The ONPO shows wide variation in oilseed rape, whereas the underlying genes and mechanisms are poorly known. The present study performed the genetic, physiological and transcriptomic analyses of ovule number per ovary using an association panel and the extreme lines. The ONPO of 327 accessions planted in four environments showed a large variation from 19.2 to 43.8, indicating a great potential for the further genetic improvement of ovule number. The genome-wide association study (GWAS) identified a total of 43 significant SNP markers. Further, these SNPs were integrated into 18 association loci, which were distributed on chromosomes A01, A03, A06, A07, A09, C01, C03, C06, C07, and C09, explaining 4.3–11.5% of the phenotypic variance. The ONPO decreased as their appearance order on the inflorescence and was associated with the level of several types of endogenous phytohormones but not related to leaf area and photosynthetic rate. Comparative transcriptomic analysis identified a total of 4,449 DEGs enriched in 30 classes, including DNA, RNA, protein, signaling, transport, development, cell wall, lipid metabolism, and secondary metabolism. Nearly half of DEGs were involved in the known pathways in regulating ovule number, of which 12 were homologous to know ovule number regulating genes, indicating a strong link between the identified DEGs and ovule number. A total of 73 DEGs were located within the genomic regions of association loci, of which six were identified as candidates based on functional annotation. These results provide useful information for the further genetic improvement of ovule and seed number in oilseed rape.