Endocrine disrupting chemicals (EDCs) are defined as exogenous chemicals, or mixture of chemicals, that can interfere with any aspect of hormone action. The field of endocrine disruption is historically rooted in wildlife biology and reproductive endocrinology where EDCs are demonstrated contributors to infertility, premature puberty, endometriosis, and other disorders. Recently, EDCs have been implicated in metabolic syndrome and obesity. Adipose tissue is a true endocrine organ and, therefore, an organ which is highly susceptible to disturbance by EDCs. A subset of EDCs, called “obesogens” promote adiposity by altering programming of fat cell development, increasing energy storage in fat tissue, and interfering with neuroendocrine control of appetite and satiety. Obesity adds more than $200 billion to U.S. healthcare costs and the number of obese individuals continues to increase. Hence, there is an urgent, unmet need to understand the mechanisms underlying how exposures to certain EDCs may predispose our population to be obese. In this review, we discuss the history of obesogen discovery from its origins in reproductive biology to its latest role in the transgenerational inheritance of obesity in mice. We discuss the development of adipose tissue in an embryo, maintenance of adipocyte number in adults, how EDC disruption programs stem cells to preferentially make more adipocytes, the mechanisms by which chemicals can permanently alter the germline epigenome, and whether there are barriers to EDCs in the gametes.