The purpose of this study is to understand the relationship among leisure physical activity, sedentary lifestyle, physical fitness, and happiness in healthy elderly adults aged over 65 years old in Taiwan. Data were recruited from the National Physical Fitness Survey in Taiwan, which was proposed in the Project on the Establishment of Physical Fitness Testing Stations by the Sports Administration of the Ministry of Education. Participants were recruited from fitness testing stations set up in 22 counties and cities from October 2015 to May 2016. A total of 20,111 healthy older adults aged 65–102 years were recruited as research participants. The fitness testing procedure was described to all participants, who were provided with a standardized structured questionnaire. Participants’ data included sex, city or county of residence, living status (living together with others or living alone), education level, and income. Physical fitness testing was conducted in accordance with The Fitness Guide for Older Adults published by the Sports Administration of the Ministry of Education. The testing involved cardiorespiratory endurance, muscle strength, muscle endurance, flexibility, balance, and body composition. The t-test was used to evaluate the differences between continuous and grade variables under the two classification variables of sex, city or county of residence, and living status. We used the MARS (multivariate adaptive regression splines) model to analyze the effects of physical fitness variables and leisure physical activity variables on happiness. Among healthy elderly adults, sex, age, living status, body mass index, and leisure physical activity habits proved to be related to happiness. Aerobic endurance (2-min step test), muscular strength and endurance (30-s arm curl and 30-s chair stand tests), flexibility (back stretch and chair sit-and-reach tests), and balance ability (8-foot up-and-go tests and one-leg stance with eyes open tests) were found to be related to happiness. The results of this study indicate that increased physical activity and intensity, as well as physical fitness performance, are associated with improved happiness.