BackgroundRecently, we found that berberine (BBR) exerts anti-acute myeloid leukemia activity, particularly toward high-risk and relapsed/refractory acute myeloid leukemia MV4-11 cells in vitro. However, the poor water solubility and low bioavailability observed with oral BBR administration has limited its clinical use. Therefore, we design and develop a novel oil-in-water self-nanoemulsifying system for BBR (BBR SNE) to improve oral bioavailability and enhance BBR efficacy against acute myeloid leukemia by greatly improving its solubility.ResultsThis system (size 23.50 ± 1.67 nm, zeta potential − 3.35 ± 0.03 mV) was prepared with RH40 (surfactant), 1,2-propanediol (co-surfactant), squalene (oil) and BBR using low-energy emulsification methods. The system loaded BBR successfully according to thermal gravimetric, differential scanning calorimetry, and Fourier transform infrared spectroscopy analyses. The release profile results showed that BBR SNE released BBR more slowly than BBR solution. The relative oral bioavailability of this novel system in rabbits was significantly enhanced by 3.41-fold over that of BBR. Furthermore, Caco-2 cell monolayer transport studies showed that this system could help enhance permeation and prevent efflux of BBR. Importantly, mice with BBR SNE treatment had significantly longer survival time than BBR-treated mice (P < 0.001) in an MV4-11 engrafted leukemia murine model.ConclusionsThese studies confirmed that BBR SNE is a promising therapy for acute myeloid leukemia.Electronic supplementary materialThe online version of this article (10.1186/s12951-018-0402-x) contains supplementary material, which is available to authorized users.