Osteoprotegerin (OPG), a secreted decoy receptor for receptor activator of nuclear factor B ligand (RANKL), plays an essential role in regulating bone resorption. While much is known about the function of the N-terminal domains of OPG, which is responsible for binding to RANKL, the exact biological functions of the three C-terminal domains of OPG remain uncertain. We have previously shown that one likely function of the C-terminal domains of OPG is to bind cell surface heparan sulfate (HS), but the in vivo evidence was lacking. To investigate the biological significance of OPG–HS interaction in bone remodeling, we created OPG knock-in mice (opgAAA). The mutated OPG is incapable of binding to HS but binds RANKL normally. Surprisingly, opgAAA/AAA mice displayed a severe osteoporotic phenotype that is very similar to opg-null mice, suggesting that the antiresorption activity of OPG requires HS. Mechanistically, we propose that the HS immobilizes secreted OPG at the surface of osteoblasts lineage cells, which facilitates binding of OPG to membrane-anchored RANKL. To further support this model, we altered the structure of osteoblast HS genetically to make it incapable of binding to OPG. Interestingly, osteocalcin-Cre;Hs2stf/f mice also displayed osteoporotic phenotype with similar severity to opgAAA/AAA mice. Combined, our data provide strong genetic evidence that OPG–HS interaction is indispensable for normal bone homeostasis.