Obesity rates are climbing, representing a confounding and contributing factor to many disease states, including cancer. With respect to breast cancer, obesity plays a prominent role in the etiology of this disease, with certain subtypes such as triple-negative breast cancer having a strong correlation between obesity and poor outcomes. Therefore, it is critical to examine the obesity-related alterations to the normal stroma and the tumor microenvironment (TME). Adipocytes and adipose stem cells (ASCs) are major components of breast tissue stroma that have essential functions in both physiological and pathological states, including energy storage and metabolic homeostasis, physical support of breast epithelial cells, and directing inflammatory and wound healing responses through secreted factors. However, these processes can become dysregulated in both metabolic disorders, such as obesity and also in the context of breast cancer. Given the well-established obesity-neoplasia axis, it is critical to understand how interactions between different cell types in the tumor microenvironment, including adipocytes and ASCs, govern carcinogenesis, tumorigenesis, and ultimately metastasis. ASCs and adipocytes have multifactorial roles in cancer progression; however, due to the plastic nature of these cells, they also have a role in regenerative medicine, making them promising tools for tissue engineering. At the physiological level, the interactions between obesity and breast cancer have been examined; here, we will delineate the mechanisms that regulate ASCs and adipocytes in these different contexts through interactions between cancer cells, immune cells, and other cell types present in the tumor microenvironment. We will define the current state of understanding of how adipocytes and ASCs contribute to tumor progression through their role in the tumor microenvironment and how this is altered in the context of obesity. We will also introduce recent developments in utilizing adipocytes and ASCs in novel approaches to breast reconstruction and regenerative medicine.