Background: Incorporating physical activity (PA) has been a challenge for health care professionals working with anorexia nervosa (AN) patients. This has contributed to partial physical fitness (PFit) recovery that persists after weight restoration. Objective: This cross-sectional study aimed to examine the relationships between PA, sedentary behaviors, PFit, and quality of life (QoL) in a group of adolescents after hospitalization. Methods: QoL, PA, and sedentary behaviors were measured using the Health-Questionnaire Short-Form 36 (SF-36) and accelerometers, while PFit was assessed through cardiorespiratory fitness, body composition (anthropometry), and strength (six repetition maximum) tests in a total of 63 patients. Results: Light-PA (LPA), moderate-PA (MPA), moderate-to-vigorous-PA (MVPA), and relative sedentary time (%ST) did not meet the recommendations (p < 0.001). Only 22% of the patients met MVPA criteria, and ~82% exceeded %ST. SF-36 scores were lower than normative values except the physical component scale. Absolute cardiorespiratory fitness was reduced (p < 0.001) in 84% of the patients, and was positively associated to body weight, body mass index (BMI), circumferences, and muscle areas. Additional positive significant relationships were found between QoL, muscular strength, and body composition, and negative associations between vigorous-to-very vigorous PA and BMI, skinfolds, and percent body fat. Regression analyses revealed lower body strength as an explanatory factor for improved QoL (OR 1.03, 95%CI 1.00–1.07). Conclusions: PFit and QoL scores are poor after hospitalization. LPA, MPA, and MVPA do not meet recommendations. PFit management—with emphasis on improving muscular fitness—may be a valuable strategy for QoL improvement in AN after hospitalization.