Gymnema sylvestre (Retz.) Schult is a multi-purpose traditional medicine that has long been used for the treatment of various diseases. To discover the potential bioactive composition of G. sylvestre, a chemical investigation was thus performed. In this research, four new C21 steroidal glycosides sylvepregosides A-D (1–4) were isolated along with four known compounds, gymnepregoside H (5), deacetylkidjoladinin (6), gymnepregoside G (7) and gymnepregoside I (8), from the ethyl acetate fraction of G. sylvestre. The structures of the new compounds were established by extensive 1D and 2D nuclear magnetic resonance (NMR) spectra with mass spectroscopy data. Compounds 1–6 promoted glucose uptake by the range of 1.10- to 2.37-fold, respectively. Compound 1 showed the most potent glucose uptake, with 1.37-fold enhancement. Further study showed that compounds 1 and 5 could promote GLUT-4 fusion with the plasma membrane in L6 cells. The result attained in this study indicated that the separation and characterization of these compounds play an important role in the research and development of new anti-diabetic drugs and pharmaceutical industry.