Traditional Chinese medication (TCM) has analgesic and anti-inflammatory effects in patients with knee osteoarthritis (OA). We conducted the first systematic review of the best quantitative and qualitative evidence currently available in order to evaluate the effectiveness of TCM in relieving pain in knee OA. A comprehensive literature search was conducted using three English and four Chinese biomedical databases from their inception through March 1, 2015. We included randomized controlled trials of TCM for knee OA with intervention durations of at least two weeks. The effects of TCM on pain and other clinical symptoms were measured with the visual analog scale (VAS) and Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC). The total effectiveness rate, which was used to assess overall pain, physical performance and wellness, was also measured. Two researchers independently extracted data on study design, population characteristics, duration, intervention, outcomes, risk of bias, and primary results. We performed a random-effects meta-analysis when appropriate. We also explored factors that could explain the heterogeneity by conducting subgroup and meta-regression analyses. Twenty-three studies, totaling 2362 subjects, met the eligibility criteria. Treatments were formulated with an average of 8 Chinese herbs and were prescribed based on the traditional Chinese diagnostic method of syndrome differentiation. The mean treatment duration was seven weeks, with oral administration occurring one to three times a day. Compared with non-steroidal anti-inflammatory drugs and intra-articular hyaluronate injections, 18 of the studies showed significantly improved VAS pain scores (Mean Difference [MD] = 0.56; 95% confidence interval [CI], 0.18 to 0.94; p = 0.004), six of the studies showed significantly improved WOMAC pain subscale scores (MD = 2.23; 95% CI, 0.56 to 3.91; p = 0.009), and 16 of the trials showed significantly improved total effectiveness rates (risk ratio = 1.12; 95% CI, 1.05 to 1.19; p = 0.0003). In addition, TCM showed a lower risk of adverse events than standard western treatments. This evidence suggests that TCM is safe and effective for improving pain, function, and wellness in treatments of knee OA. However, there is inherent clinical heterogeneity (diverse TCM formulations, controls, and treatment regimens) among the included trials. Despite these limitations, the potential analgesic effects of TCM warrant further methodologically rigorous research to determine the clinical implications of TCM on pain management in knee OA.