Background Knee OA is a chronic disease associated with significant morbidity and economic cost. The efficacy of acupuncture in addition to traditional physical therapy has received little study. Objective To compare the efficacy and safety of integrating a standardized true acupuncture protocol versus non-penetrating acupuncture into exercise-based physical therapy (EPT). Methods This was a randomized, double-blind, controlled trial at 3 physical therapy centers in Philadelphia, PA. We studied 214 patients (66% African-American) with at least 6 months of chronic knee pain and X-ray confirmed Kellgren scores of 2 or 3. Patients received 12 sessions of acupuncture directly following EPT over 6–12 weeks. Acupuncture was performed at the same 9 points dictated by the Traditional Chinese “Bi” syndrome approach to knee pain, using either standard needles or Streitberger non-skin puncturing needles. The primary outcome was the proportion of patients with at least a 36% improvement in Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) score at 12 weeks. Results Both treatment groups showed improvement from combined therapy with no difference between true (31.6%) and non-penetrating acupuncture (30.3%) in WOMAC response rate (p=0.5) or report of minor adverse events. A multivariable logistic regression prediction model identified an association between a positive expectation of relief from acupuncture and reported improvement. No differences were noted by race, sex, or age. Conclusion Puncturing acupuncture needles did not perform any better than non-puncturing needles integrated with EPT. Whether EPT, acupuncture, or other factors accounted for any improvement noted in both groups could not be determined in this study. Expectation for relief was a predictor of reported benefit.