Sarcopenia is a systemic skeletal muscle disease that is more prevalent in older adults. The role of exercise in improving the disease has been demonstrated. However, due to the variety of exercise modalities, it is not clear what type of exercise provides the best benefit. The aim of this meta-analysis was to analyze the effects of different exercise modalities on grip strength, appendicular skeletal muscle index, skeletal muscle index, and knee extensor strength in elderly patients with sarcopenia. The protocol for this evaluation was registered on the PROSPERO website and the databases PubMed, WOS, Cochrane Library, and Embase were searched. Thirteen studies were included in the analysis. The results showed that exercise interventions had positive effects on grip strength and knee extension muscle strength, with resistance training being the most effective. There was no significant improvement in appendicular skeletal muscle index or skeletal muscle index. This study still has limitations. For example, age group and exercise duration were not considered. Future studies should further explore benefits in age groups as well as other relevant outcome indicators.