Obesity is associated with an increased risk of breast cancer, which is the most common cancer in women worldwide (excluding non-melanoma skin cancer). Furthermore, breast cancer patients with obesity have an impaired prognosis. Adipose tissue is abundant in the breast. Therefore, breast cancer develops in an adipose-rich environment. During obesity, changes in the local environment in the breast occur which are associated with breast cancer. A shift towards a pro-inflammatory state is seen, resulting in altered levels of cytokines and immune cells. Levels of adipokines, such as leptin, adiponectin, and resistin, are changed. Aromatase activity rises, resulting in higher levels of potent estrogen in the breast. Lastly, remodeling of the extracellular matrix takes place. In this review, we address the current knowledge on the changes in the breast adipose tissue in obesity associated with breast cancer initiation and progression. We aim to identify obesity-associated biomarkers in the breast involved in the interplay between obesity and breast cancer. Hereby, we can improve identification of women with obesity with an increased risk of breast cancer and an impaired prognosis. Studies investigating mammary adipocytes and breast adipose tissue in women with obesity versus women without obesity are, however, sparse and further research is needed.