Human sodium-glucose cotransporter 2 (hSGLT2) is a membrane protein responsible for glucose reabsorption from the glomerular filtrate in the proximal tubule. Inhibition of hSGLT2 has been regarded as a brand new therapeutic approach for the treatment of type 2 diabetes mellitus (T2DM) due to its non-insulin related characteristics with less side effects. Current commercially available hSGLT2 inhibitors are all C-glycoside inhibitors. Previous studies have reported that N-glycoside inhibitors have better potential to serve as new drugs due to their good metabolic stability. In addition, non-glycoside inhibitors have been shown to exhibit the capability to overcome the existing problems of current glycoside inhibitors, including low tissue permeability, poor stability and short serum half-time. Here, we aimed to discover novel N-glycoside and non-glycoside hSGLT2 inhibitors by a combination of several computational approaches. A ligand-based pharmacophore model was generated, well validated and subsequently utilized as a 3D query to identify novel hSGLT2 inhibitors from National Cancer Institute (NCI) and Traditional Chinese Medicine (TCM) databases. Finally, one N-glycoside (NSC679207) and one non-glycoside (TCM_Piperenol_A) hSGLT2 inhibitors were successfully identified, which were proven to exhibit excellent binding affinities, pharmacokinetic properties and less toxicity than the commercially available hSGLT2 inhibitor, canagliflozin, via molecular docking, ADMET prediction, molecular dynamics (MD) simulations and binding free energy calculations. All together, our results strongly suggest that these two compounds have great potential to serve as novel hSGLT2 inhibitors for the treatment of T2DM and their efficacies may be further examined by a series of in vitro and/or in vivo bioassays.