Breast cancer is the most frequently diagnosed cancer and a common cause of cancer-related death in women. It is well recognized that obesity is associated with an enhanced risk of more aggressive breast cancer as well as reduced patient survival. Adipose tissue is the major microenvironment of breast cancer. Obesity changes the composition, structure, and function of adipose tissue, which is associated with inflammation and metabolic dysfunction. Interestingly, adipose tissue is rich in ASCs/MSCs, and obesity alters the properties and functions of these cells. As a key component of the mammary stroma, ASCs play essential roles in the breast cancer microenvironment. The crosstalk between ASCs and breast cancer cells is multilateral and can occur both directly through cell–cell contact and indirectly via the secretome released by ASC/MSC, which is considered to be the main effector of their supportive, angiogenic, and immunomodulatory functions. In this narrative review, we aim to address the impact of obesity on ASCs/MSCs, summarize the current knowledge regarding the potential pathological roles of ASCs/MSCs in the development of breast cancer, discuss related molecular mechanisms, underline the possible clinical significance, and highlight related research perspectives. In particular, we underscore the roles of ASCs/MSCs in breast cancer cell progression, including proliferation and survival, angiogenesis, migration and invasion, the epithelial–mesenchymal transition, cancer stem cell development, immune evasion, therapy resistance, and the potential impact of breast cancer cells on ASCS/MSCs by educating them to become cancer-associated fibroblasts. We conclude that ASCs/MSCs, especially obese ASCs/MSCs, may be key players in the breast cancer microenvironment. Targeting these cells may provide a new path of effective breast cancer treatment.