The expansion of ageing population has gained much public attention on the importance of healthy and successful ageing, which is absence of major chronic diseases, preserved physiological and cognitive functioning and active engagement with life. Previous studies have found there was a significant correlation between physical fitness with cognition. However, the relationship between physical fitness with successful and unsuccessful cognitive ageing groups are very limited. This study was aimed to identify the significant physical fitness components that contribute in reducing risk of cognitive decline represented as different cognitive ageing groups. A total of 300 community-based elderly aged 60 and above from the states of Selangor, Perak and Kelantan were recruited using multistage random sampling method in this cross-sectional study. Cognitive function of subjects was categorized into three groups, namely Mild Cognitive Impairment (MCI) (n = 100), Usual Ageing (UA) (n = 100) and Successful Ageing (SA) (n = 100) based on defined criteria. Senior Fitness Tests included 2-minute step, handgrip strength, chair stand, chair sit-and-reach, 8 foot up-and-go and back scratch were measured to determine the cardiorespiratory fitness; muscle strength; agility and flexibility of subjects. SA group had significantly better performance than non-SA groups in all fitness components, except for chair sit-and-reach. After controlling for age, gender, education years and smoking status, handgrip strength and chair stand tests were associated with a reduced risk of MCI by 7% [OR: 0.93, 95% C.I: 0.88-0.99, p < 0.05] and 15% [OR: 0.85, 95% C.I: 0.75-0.95, p < 0.01], respectively. These findings suggest that older adults with higher upper and lower body muscular strength could serve as protective factors for cognitive impairment. Further research is warranted to evaluate the mechanism of physical and cognitive decline such as Motoric Cognitive Risk Syndrome (MCR) in more detailed for the purpose for promoting healthy and successful ageing.