Undernutrition is a health challenge due to an expanding older population. The aims of the study were to assess the prevalence and determinants of undernutrition and, associated factors of low muscle and high fat mass among older men and women in the Colombo district of Sri Lanka. A cross sectional study was conducted using a multistage cluster sampling technique. Undernutrition was defined based on anthropometry and body composition assessed using bio-electrical impedance. Sex-specific multivariable logistic regression analyses were conducted. Of 800 participants (30.6% men), 35.3% were undernourished. The factors significantly associated with undernutrition among older women were hypertension with an adjusted odds ratio (aOR) (1.97; 1.36–2.88) and musculoskeletal disabilities aOR (2.19; 1.36–3.53). Among women, age ≥ 70 (1.79; 1.18–3.34) and diabetes (1.77; 1.10–2.84) were associated with low muscle mass and age ≥ 70 (2.05; 1.21–3.47), diabetes (2.20; 1.35–3.59) and disability in chewing (2.39; 1.30–4.40) were associated with high fat mass. Among men, age ≥ 70 years, no/up to grade 5 education, diabetes, visual disability, little/no responsibility in food shopping and not getting nutritional advice from media were associated with reduced odds of low muscle mass and no/up to grade 5 school education, disability in chewing and little/no responsibility in food shopping were associated with reduced odds of high fat mass. Undernutrition among older people is common in Sri Lanka. We have identified key factors associated with low muscle mass and high fat mass in this setting. Given the potential consequences of these conditions, our study provides potential targets for prevention of undernutrition and sarcopenic obesity.