Several studies investigated the role of physical activity in atrial fibrillation (AF), but the results remain controversial. We aimed to estimate the association between physical activity and incident AF, as well as to determine whether a sex difference existed. We systematically retrieved relevant studies from Cochrane Library, PubMed, and ScienceDirect through December 1, 2015. Data were abstracted from eligible studies and effect estimates pooled using a random-effects model. Thirteen prospective studies fulfilled inclusion criteria. For primary analysis, neither total physical activity exposure (relative risk [RR]: 0.98, 95% confidence interval [CI]: 0.90-1.06, P = 0.62) nor intensive physical activity (RR: 1.07, 95% CI: 0.93-1.25, P = 0.41) was associated with a significant increased risk of AF. In the country-stratified analysis, the pooled results were not significantly changed. However, in the sex-stratified analysis, total physical activity exposure was associated with an increased risk of AF in men (RR: 1.18, 95% CI: 1.02-1.37), especially at age <50 years (RR: 1.58, 95% CI: 1.28-1.95), with a significantly reduced risk of AF in women (RR: 0.92, 95% CI: 0.87-0.97). Additionally, male individuals with intensive physical activity had a slightly higher (although statistically nonsignificant) risk of developing AF (RR: 1.12, 95% CI: 0.99-1.28), but there was a significantly reduced risk of incident AF in women (RR: 0.92, 95% CI: 0.86-0.98). Published literature supports a sex difference in the association between physical activity and incident AF. Increasing physical activity is probably associated with an increased risk of AF in men and a decreased risk in women.