Elderberry (Sambucus nigra L.) is a widespread deciduous shrub, of which the fruits (elderberries) are used in the food industry to produce different types of dietary supplement products. These berries have been found to show multiple bioactivities, including antidiabetic, anti-infective, antineoplastic, anti-obesity, and antioxidant activities. An elderberry extract product, Sambucol®, has also been used clinically for the treatment of viral respiratory infections. As the major components, phenolic compounds, such as simple phenolic acids, anthocyanins and other flavonoids, and tannins, show promising pharmacological effects that could account for the bioactivities observed for elderberries. Based on these components, salicylic acid and its acetate derivative, aspirin, have long been used for the treatment of different disorders. Dapagliflozin, an FDA-approved antidiabetic drug, has been developed based on the conclusions obtained from a structure–activity relationship study for a simple hydrolyzable tannin, β-pentagalloylglucoside (β-PGG). Thus, the present review focuses on the development of therapeutic agents from elderberries and their small-molecule secondary metabolites. It is hoped that this contribution will support future investigations on elderberries.