Colorectal cancer (CRC) is among the major threatening diseases worldwide, being the third most common cancer, and a leading cause of death, with a global incidence expected to increase in the coming years. Enhanced adiposity, particularly visceral fat, is a major risk factor for the development of several tumours, including CRC, and represents an important indicator of incidence, survival, prognosis, recurrence rates, and response to therapy. The obesity-associated low-grade chronic inflammation is thought to be a key determinant in CRC development, with the adipocytes and the adipose tissue (AT) playing a significant role in the integration of diet-related endocrine, metabolic, and inflammatory signals. Furthermore, AT infiltrating immune cells contribute to local and systemic inflammation by affecting immune and cancer cell functions through the release of soluble mediators. Among the factors introduced with diet and enriched in AT, fatty acids (FA) represent major players in inflammation and are able to deeply regulate AT homeostasis and immune cell function through gene expression regulation and by modulating the activity of several transcription factors (TF). This review summarizes human studies on the effects of dietary FA on AT homeostasis and immune cell functions, highlighting the molecular pathways and TF involved. The relevance of FA balance in linking diet, AT inflammation, and CRC is also discussed. Original and review articles were searched in PubMed without temporal limitation up to March 2021, by using fatty acid as a keyword in combination with diet, obesity, colorectal cancer, inflammation, adipose tissue, immune cells, and transcription factors.