Sarcopenia has become important to the public health with the increase in the aging population in society. However, the therapeutic effects of conventional approaches, including pharmacotherapy, exercise, and nutritional intervention, are far from satisfactory. Chinese herbal medicine is a new treatment format with interesting possibilities in sarcopenia has been widely practiced. The study aims to explore the effectiveness of Chinese herbal medicine in sarcopenia. We comprehensively searched the following electronic databases: Medline, EMBASE, APA PsycInfo, Cochrane Library, Web of Science, PubMed, and Chinese database from the establishment of the database to December 2022 (no language restrictions). Randomized controlled clinical studies on the use of Chinese herbal medicine in sarcopenia were selected in compliance with PRISMA guidelines. Review Manager and Stata were used for statistical analysis and the mean difference and standardized mean difference were adopted. Of 277 identified studies, 17 were eligible and included in our analysis (N = 1440 participants). The results showed that Chinese herbal medicine can improve total efficiency (RR = 1.29, 95% CI [1.21, 1.36], p < 0.00001) in sarcopenia and enhance muscle mass (SMD = 1.02, 95% CI [0.55, 1.50], p < 0.0001), and muscle strength measured by grip strength (SMD = 0.66, 95% CI [0.36, 0.96], p < 0.0001), measured by 60°/s knee extension peak TQ (MD = 5.63, 95% CI [−0.30, 11.57], p = 0.06) and muscle function measured by 6‐meter walking speed (SMD = 1.34, 95% CI [0.60, 2.08], p = 0.0004), measured by the short physical performance battery of 1.50%, 95% CI (1.05, 1.95), measured by the EuroQoL 5‐dimension of (SMD = 0.27, 95% CI [−0.10, 0.65], p = 0.16), suggesting that Chinese herbal medicine alone or combined with conventional treatment has ameliorating effect on sarcopenia. Chinese herbal medicine is a potential therapeutic strategy in sarcopenia. The funnel plot and Egger's test indicated publication bias. To confirm our conclusions, further high‐quality studies should be conducted.