Background: Physical activity (PA) is a health determinant among middle-aged and older adults. In contrast, poor health is expected to have a negative impact on PA. This study sought to assess to what extent specific International Classification of Functioning, Disability and Health (ICF) health components were associated with PA among older adults. Methods: We used a sample of 864 persons aged ≥50 years, positively screened for disability or cognition in a cross-sectional community survey in Spain. Weekly energy expenditure during PA was measured with the Yale Physical Activity Survey (YPAS) scale. The associations between body function impairment, health conditions or World Health Organization Disability Assessment Schedule (WHODAS 2.0) disability scores and energy expenditure were quantified using negative-binomial regression, and expressed in terms of adjusted mean ratios (aMRs). Results: Mean energy expenditure was 4542 Kcal/week. A lower weekly energy expenditure was associated with: severe/extreme impairment of mental functions, aMR 0.38, 95% confidence interval, CI (0.21-0.68), and neuromusculoskeletal and movement functions, aMR 0.50 (0.35-0.72); WHODAS 2.0 disability, aMR 0.55 (0.34-0.91); dementia, aMR 0.45 (0.31-0.66); and heart failure, aMR 0.54 (0.34-0.87). In contrast, people with arthritis/ osteoarthritis had a higher energy expenditure, aMR 1.27 (1.07-1.51). Conclusion: Our results suggest that there is a strong relationship between selected body function impairments, mainly mental, and PA. Although more research is needed to fully understand causal relationships, strategies to improve PA among the elderly may require targeting mental, neuromusculoskeletal and movement functions, disability determinants (including barriers), and specific approaches for persons with dementia or heart failure.