Background: Diabetic autonomic neuropathy is a common complication of type 2 diabetes mellitus (T2DM), especially in patients with long-term, poorly controlled diabetes. This study investigates the effects of exercise on autonomic nervous system activity in T2DM patients over time. Methods: A literature review using MEDLINE, Embase, Cochrane Library, Scopus, and PubMed identified studies assessed via heart rate variability. Papers were categorized into three groups: immediate effects (within 60 min), short-term effects (2–3 months), and long-term effects (over 4 months). Results: Nine articles with 161 T2DM patients were included in the meta-analysis. RMSSD changes after exercise were −4.3 (p = 0.227), 8.14 (p < 0.001), and 4.17 (p = 0.002) for the immediate, short-term, and long-term groups, respectively. LF/HF ratio changes were 0.21 (p = 0.264), −3.04 (p = 0.102), and −0.05 (p = 0.006) for the respective groups. Meta-regression indicated age, male gender, and exercise duration were associated with increased RMSSD, with coefficients of 2.36 (p = 0.001), 13.76 (p = 0.008), and 1.50 (p = 0.007), respectively. Age positively correlated with the LF/HF ratio, with a coefficient of 0.049 (p = 0.048). Conclusions: Regular exercise (≥3 times per week) for over 2 months increases parasympathetic activity in T2DM patients, while sympathetic activity decreases significantly after 4 months. Further study is needed to validate these findings.