Low bone mineral density (BMD) is a frequent complication in anorexia nervosa (AN). There are controversial points of views regarding the restoration of bone mineralization after recovery in AN. We aimed to assess changes of BMD at 3 years in patients with AN and to explore the relationships between body composition, physical activity, and BMD. Patients with AN were included from 2009 to 2011 in a first visit (T0) with evaluation of weight, height, body mass index (BMI), body composition [fat mass (FM) and fat-free mass], and BMD. Those who had low BMD, either osteoporosis or osteopenia, were admitted in a second visit (T1) to carry out a new bone densitometry examination and body composition; they were also asked for their physical activity. At T0, our study involved 160 patients. Low BMD was observed in 53.6% of them and significant factors associated with demineralization were lower BMIs (16.5 ± 2.1 vs 17.3 ± 2.3 kg/m2, p = 0.01) and higher duration of AN (11.4 ± 10.5 vs 6.4 ± 6.5 years, p = 0.001). At 3 years follow-up (T1), 42 patients were involved and no significant changes in BMD were observed despite body weight increase (3.8 ± 6.1 kg). Interestingly, FM gain was a significant factor associated with BMD improvement at follow-up (8.0 ± 9.1 vs 3.0 ± 3.5 kg, p = 0.02). Our findings suggest that the restoration of normal bone values is not related to the increase of body weight, at least after 3 years. FM seems to play an important role in the pathophysiological mechanism of osteoporosis and osteopenia in AN.