The aim of this systematic review and meta-analysis is to determine the role of physical activity as a conservative treatment for older people with knee or hip osteoarthritis. The effect on pain, physical function, stiffness, quality of life, and dynamic balance of Aquatic Exercise, Land-based Exercise, and Sports were compared in a specific population composed of osteoarthritic patients aged 65 or over. A systematic search using Pubmed-Medline, Google Scholar, and the Cochrane Library was carried out to select randomized clinical trials, observational studies, or case series that evaluated outcome measures after physical activity. Twenty randomized controlled trials (RCTs) and two case series were included in this review. Four trials were at low risk of bias (A), 12 at unclear risk of bias (B), and four at high risk of bias (C). Compared to controls, Aquatic Exercise, Land-based Exercise, Tai Chi, and Yoga showed a small to high effect for improving pain, physical function, quality of life, and stiffness. Active exercise and sport are effective to improve pain and physical function in elderly people with osteoarthritis. Nevertheless, further studies are required to validate the use of land-based exercise, aquatic exercise, or sport to treat the symptoms of older adults that suffer from knee and hip osteoarthritis.