Repeated weight fluctuation has been proposed as a potential risk factor for increasing morbidity and mortality including cancer. We aimed to investigate the association between body weight variability (BWV) and all cancer and site-specific cancer incidence and the impact of smoking on these associations. A total of 1,759,848 cancer-free male subjects who had their weight measured at least 5 times from the National Health Insurance Service-Health Screening Cohort from 2002 to 2011 were included and followed up until 2015. BWV was defined as the average absolute difference between successive values (ASV). The risk of cancer and site-specific cancer from BWV was identified using Cox proportional hazards regression analysis using hazard ratios (HRs) and 95% confidence intervals (CIs) adjusted for potential confounders including weight, and stratified analysis was also conducted according to smoking status. During the 7,015,413 person-years of follow-up, 11,494 patients (0.65%) developed new-onset cancers. BWV was associated with a higher risk of all cancers after adjustment for confounders. The highest BWV quintile group compared to the lowest had greater risks of all cancers and site-specific cancers including lung, liver, and prostate cancer (HR 1.22, 95% CI 1.15–1.30; HR 1.22, 95% CI 1.07–1.39; HR 1.46, 95% CI 1.19–1.81; HR 1.36, 95% CI 1.15–1.62, in all cancers, lung, liver and prostate cancer, respectively). Due to small number of cancer occurrence, the risk of kidney cancer was increased, but statistically insignificant (HR 1.38, 95% CI 0.91–2.10). Similar results were observed in noncurrent smokers. However, in current smokers, the risks of all cancers and only prostate cancer were significantly increased in the highest BWV quintile group (HR 1.19, 95% CI 1.09–1.31; HR 1.51, 95% CI 1.08–2.11). The risk of kidney cancer also increased in this group, although the finding was not statistically significant (HR 1.77, 95% CI 0.87–3.63) This study suggested BWV is an independent risk factor for cancer in men, especially in lung, liver, and prostate cancer, but evidence was weaker in kidney cancer. This association remained significant only in prostate cancer in current smokers.