This study aimed to analyze the efficacy of exercise interventions on muscle strength, muscle mass, and physical performance in older adults with sarcopenia. Randomized controlled studies assessing exercise effects on sarcopenia were searched in Web of Science, PubMed, Cochrane Library, ProQuest, EBSCOhost, Scopus, EMBASE, and VIP and CNKI up to 31 March 2022. Data were expressed as weighted/standardized mean difference (MD/SMD) with 95% confidence intervals (CI). I2 index was employed for heterogeneity. The initial search identified 5379 studies, and 23 studies involving 1252 participants met the inclusion criteria for further analysis. Results revealed that exercise interventions can significantly improve grip strength (MD = 2.38, 95%CI = 1.33–3.43), knee extension strength (SMD = 0.50, 95%CI = 0.36 ~ 0.64), muscle mass of lower extremities (MD = 0.28, 95%CI = 0.01–0.56), walking speed (SMD = 0.88, 95%CI = 0.49–1.27), and functional mobility (MD = −1.77, 95%CI = −2.11–−1.42) among older adults with sarcopenia. No significant exercise effects were found on fat-free muscle mass, appendicular muscle mass, skeletal muscle mass, and muscle mass of the upper extremities. The results of subgroup analysis indicated that both resistance training and multicomponent exercise could significantly increase the muscle strength, while aerobic exercise did not. The findings suggest that exercise intervention can effectively improve muscle function and physical performance in older adults with sarcopenia, but has limited effects on the muscle mass of the upper extremities. In addition, it is highly recommended to apply group-based and supervised resistance training and multicomponent exercise in the prevention and treatment of sarcopenia among the older population.