Rheumatoid arthritis (RA) is a common systematic, chronic inflammatory, autoimmune, and polyarticular disease, causing a range of clinical manifestations, including joint swelling, redness, pain, stiffness, fatigue, decreased quality of life, progressive disability, cardiovascular problems, and other comorbidities. Strong evidence has shown that exercise is effective for RA treatment in various clinical domains. Exercise training for relatively longer periods (e.g., ≥ 12 weeks) can decrease disease activity of RA. However, the mechanism underlying the effectiveness of exercise in reducing RA disease activity remains unclear. This review first summarizes and highlights the effectiveness of exercise in RA treatment. Then, we integrate current evidence and propose biological mechanisms responsible for the potential effects of exercise on immune cells and immunity, inflammatory response, matrix metalloproteinases, oxidative stress, and epigenetic regulation. However, a large body of evidence was obtained from the non-RA populations. Future studies are needed to further examine the proposed biological mechanisms responsible for the effectiveness of exercise in decreasing disease activity in RA populations. Such knowledge will contribute to the basic science and strengthen the scientific basis of the prescription of exercise therapy for RA in the clinical routine.