Glucose, fructose, and galactose are widely used in the food industry as sweeteners and food additives. The over-consumption of these carbohydrates has been identified as a possible trigger of non-communicable diseases. These include insulin resistance, obesity, and type 2 diabetes. These sugars induce an energy overload with consequent adipose tissue (AT) expansion, contributing to the development of obesity. Furthermore, a common feature of these non-communicable diseases is the detrimental, chronic, low-grade inflammation contributing to their onset. In the present review, we identify the most widely used dietary free sugars and their direct impacts on AT metabolism and inflammation, as well as their involvement in systemic inflammation and effects on the immune cell phenotype and function. Additionally, we discuss the capacity of the free sugars to induce immune modulation, enhancing inflammation, an underlying hallmark of insulin resistance, obesity, and T2DM. Dietary sugars have an important and deleterious metabolic impact on AT and also on immune cells. More research is needed to effectively understand the impact of chronic exposure to high levels of individual or combined sugars on metabolism, with the impact on immunomodulation being especially important.