Human sensation for sweet tastes and the thus resulting over-consumption of sugar in recent decades has led to an increasing number of people suffering from caries, diabetes, and obesity. Therefore, a demand for sugar substitutes has arisen, which increasingly has turned towards natural sweeteners over the last 20 years. In the same period, thanks to advances in bioinformatics and structural biology, understanding of the sweet taste receptor and its different binding sites has made significant progress, thus explaining the various chemical structures found for sweet tasting molecules. The present review summarizes the data on natural sweeteners and their most important (semi-synthetic) derivatives until the end of 2019 and discusses their structure–activity relationships, with an emphasis on small-molecule high-intensity sweeteners.