Leisure‐time physical activity (LTPA) decreases the risk of dementia, whereas occupational physical activity (OPA) possibly increases the risk. Yet, previous findings are mixed. We therefore aimed to investigate the effect of LTPA and OPA, respectively, on dementia among men and women. In this observational, longitudinal study, we used data from the second wave of a population‐based cohort from the municipality of Copenhagen as baseline. Data were collected in 1981–1983, and 10 343 participants were followed until the end of 2016. LTPA and OPA were self‐reported, and information on dementia diagnoses and redemption of dementia medication was obtained at an individual level from national health registers. We used Poisson regression to analyze the association between LTPA/OPA and dementia and adjusted for self‐reported age, socioeconomic factors, stress, and cardiovascular risk factors (smoking, alcohol, body mass index, and blood pressure). A higher level of LTPA was associated with a lower dementia risk among men, but we found no clear association among women. OPA and dementia were not associated among men, but occupationally active women who reported OPA in terms of walking, lifting, and heavy work had a higher risk of dementia than women with sedentary jobs. This study supported earlier findings of a protective effect of LTPA on dementia among men. Women in physically demanding jobs possibly have a higher risk of dementia, yet this finding warrants further investigation in future studies.