Aroma is an important economic trait of vegetable soybeans, which greatly influences their market value. The 2-acetyl-1-pyrroline (2AP) is considered as an important substance affecting the aroma of plants. Although the 2AP synthesis pathway has been resolved, the differences of the 2AP synthesis in the aromatic and non-aromatic vegetable soybeans are unknown. In this study, a broad targeted metabolome analysis including measurement of metabolites levels and gene expression levels was performed to reveal pathways of aroma formation in the two developmental stages of vegetable soybean grains [35 (S5) and 40 (S6) days after anthesis] of the ‘Zhexian No. 8′ (ZX8, non-aromatic) and ZK1754 (aromatic). The results showed that the differentially accumulated metabolites (DAMs) of the two varieties can be classified into nine main categories including flavonoids, lipids, amino acids and derivatives, saccharides and alcohols, organic acids, nucleotides and derivatives, phenolic acids, alkaloids and vitamin, which mainly contributed to their phenotypic differences. Furthermore, in combination with the 2AP synthesis pathway, the differences of amino acids and derivatives were mainly involved in the 2AP synthesis. Furthermore, 2AP precursors’ analysis revealed that the accumulation of 2AP mainly occurred from 1-pyrroline-5-carboxylate (P5C), not 4-aminobutyraldehyde (GABald). The quantitative RT-PCR showed that the associated synthetic genes were 1-pyrroline-5-carboxylate dehydrogenase (P5CDH), ∆1-pyrroline-5-carboxylate synthetase (P5CS), proline dehydrogenase (PRODH) and pyrroline-5-carboxylate reductase (P5CR), which further verified the synthetic pathway of 2AP. Furthermore, the betaine aldehyde dehydrogenase 2 (GmBADH2) mutant was not only vital for the occurrence of 2AP, but also for the synthesis of 4-aminobutyric acid (GABA) in vegetable soybean. Therefore, the differences of 2AP accumulation in aromatic and non-aromatic vegetable soybeans have been revealed, and it also provides an important theoretical basis for aromatic vegetable soybean breeding.