Adipose tissue, an energy storage and endocrine organ, is emerging as an essential player for ECM remodeling. Fibrosis is one of the hallmarks of obese adipose tissue, featuring excessive ECM deposition and enhanced collagen alignment. A variety of ECM components and ECM-related enzymes are produced by adipocytes and myofibroblasts in obese adipose tissue. Data from lineage-tracing models and a single-cell analysis indicate that adipocytes can transform or de-differentiate into myofibroblast/fibroblast-like cells. This de-differentiation process has been observed under normal tissue development and pathological conditions such as cutaneous fibrosis, wound healing, and cancer development. Accumulated evidence has demonstrated that adipocyte de-differentiation and myofibroblasts/fibroblasts play crucial roles in obesity-associated ECM remodeling and cancer progression. In this review, we summarize the recent progress in obesity-related ECM remodeling, the mechanism underlying adipocyte de-differentiation, and the function of obesity-associated ECM remodeling in cancer progression.