Background Knee osteoarthritis (KOA) is among the most common osteoarthritis diseases that affect adults older than 65 years old. Impaired neuromuscular function contributes to the development and/or progression of KOA. As a new modality in strength training, whole-body vibration (WBV) training is considered in the management of KOA. However, the inconsistent results of previous studies dampened the enthusiasm for the clinical application of WBV. Methods Eligible participants with KOA were randomly allocated to WBV, strength training (ST) and health education (HE) groups. Finally, 57 participants completed the intervention and measurements. The participants in each group were encouraged to perform the WBV training, similar strength training or health education program. These supervised interventions were performed three times per week for 8 weeks. Physical function was assessed with Timed Up and Go (TUG) and 6-min Walk Distance tests. Neuromuscular function was measured with isokinetic muscle strength and proprioception. Results All variables of physical function and neuromuscular function failed to change significantly among three groups (p > 0.05). However, compared with the baseline, the time of TUG (mean difference = 0.88 ± 0.78, 95% CI = 0.50 – 1.26, p < 0.001) and most of the variables related to isokinetic knee muscle strength (peak work of flexor at 90°/s, peak torque of extensor at 180°/s, peak work of extensor at 180°/s, peak torque of flexors at 180°/s, and peak work of flexors at 180°/s, all p < 0.05) improved significantly in WBV group. Conclusion WBV training has been recommended for strength training in elderly people and patients with musculoskeletal diseases. The 8-week WBV training under the protocol of our study was not superior to ST and HE. Hence, more clinical studies are required in the future to develop an optimal training protocol. Trial registration It was registered at Chinese Clinical Trial Registry a priori as a clinical trial (ID: ChiCTR-IOR-16009234). Registered 21 September 2016.