Immunomodulators are agents able to affect the immune system, by boosting the immune defences to improve the body reaction against infectious or exogenous injuries, or suppressing the abnormal immune response occurring in immune disorders. Moreover, immunoadjuvants can support immune system acting on nonimmune targets, thus improving the immune response. The modulation of inflammatory pathways and microbiome can also contribute to control the immune function. Some plant-based nutraceuticals have been studied as possible immunomodulating agents due to their multiple and pleiotropic effects. Being usually more tolerable than pharmacological treatments, their adjuvant contribution is approached as a desirable nutraceutical strategy. In the present review, the up to date knowledge about the immunomodulating properties of polysaccharides, fatty acids and labdane diterpenes have been analyzed, in order to give scientific basic and clinical evidence to support their practical use. Since promising evidence in preclinical studies, limited and sometimes confusing results have been highlighted in clinical trials, likely due to low methodological quality and lacking standardization. More investigations of high quality and specificity are required to describe in depth the usefulness of these plant-derived nutraceuticals in the immune system modulation, for health promoting and disease preventing purposes.