Diabetes mellitus (DM) is a serious metabolic disorder which has reached epidemic proportions all over the world. Despite tremendous developments in medicinal chemistry, traditional medicine is still commonly used for the prevention and treatment of DM. Traditional herbal medicines have been playing a major role in the management of DM in Asian countries. In particular, mulberry has commonly been utilized in China for the treatment of DM for thousands of years. In the last decade, numerous preclinical findings have suggested that mulberry is a promising therapeutic agent for the treatment of DM, and the polyhydroxylated alkaloids, flavonoids and polysaccharides from mulberry may be the potential active components. The present review systematically summarizes the chemical composition of mulberry and the pharmacological effects of different medicinal parts on DM; these effects include influences on glucose absorption, insulin (INS) production/secretion, antioxidation and anti-inflammation processes. After summarizing research findings, we discuss challenges and opportunities, and explore the direction of future research and the potential for developing mulberry into pharmaceuticals for the widespread treatment of DM.