The role of the adipocyte in the tumor microenvironment has received significant attention as a critical mediator of the obesity-cancer relationship. Current estimates indicate that 650 million adults have obesity, and thirteen cancers, including breast cancer, are estimated to be associated with obesity. Even in people with a normal body mass index, adipocytes are key players in breast cancer progression because of the proximity of tumors to mammary adipose tissue. Outside the breast microenvironment, adipocytes influence metabolic and immune function and produce numerous signaling molecules, all of which affect breast cancer development and progression. The current epidemiologic data linking obesity, and importantly adipose tissue, to breast cancer risk and prognosis, focusing on metabolic health, weight gain, and adipose distribution as underlying drivers of obesity-associated breast cancer is presented here. Bioactive factors produced by adipocytes, both normal and cancer associated, such as cytokines, growth factors, and metabolites, and the potential mechanisms through which adipocytes influence different breast cancer subtypes are highlighted.