The commitment and differentiation of bone marrow mesenchymal stem cells (MSCs) is tightly controlled by the local environment ensuring lineage differentiation balance and bone homeostasis. However, pathological conditions linked with osteoporosis have changed the bone marrow microenvironment, shifting MSCs’ fate to favor adipocytes over osteoblasts, and consequently leading to decreased bone mass with marrow fat accumulation. Multiple questions related to the underlying mechanisms remain to be answered. As recent findings have confirmed the fundamental role of the epigenetic mechanism in connecting environmental signals with gene expression and stem cell differentiation, a regulatory network in the bone marrow microenvironment, epigenetic modulation, gene expression, and MSC differentiation begins to emerge. This review discusses how pathological environmental factors affect MSCs’ fate by epigenetic modulating lineage-specific genes. We conclude that manipulating local environments and/or the epigenetic regulatory machinery that target the adipocyte differentiation pathway might be a therapeutic implication of bone loss diseases such as osteoporosis.