Background Type 2 diabetes mellitus (T2DM) is one of the most important risk factors for cardiovascular diseases, with a high economic burden on health care systems. Since gender and residency can affect people’s lifestyle and health behaviors, this study was conducted to investigate the prevalence of T2DM and its determinants by gender and residency. Methods A secondary analysis study was conducted on the survey data of the IraPEN (Iran’s Package of Essential Non-Communicable Disease) pilot program conducted in 2017 in Naghadeh County, Iran. Data of 3,691 participants aged 30–70 years from rural and urban areas of the County were included into data analysis process. Sociodemographic factors, anthropometric measurements, and cardiovascular risk factors related to T2DM were assessed. Results The overall prevalence of T2DM within the population was 13.8%, which was significantly higher among women (15.5%) than men (11.8%), and non-significantly higher in urban (14.5%) areas than rural (12.3%) areas. In both genders, age (male: OR 1.01, 95% CI: 1.00–1.03; P = 0.012; female: OR 1.03, 95% CI: 1.02–1.04; P<0.001), blood pressure (male: OR 1.77, 95% CI: 1.13–2.79; P = 0.013; female: OR 2.86, 95% CI: 2.12–3.85; P<0.001), and blood triglycerides (male: OR 1.46, 95% CI: 1.01–2.11; P = 0.04; female: OR 1.34, 95% CI: 1.02–1.77; P = 0.035) had a significant relationship with the chance of developing T2DM. Among women, a significant relationship was found between abdominal obesity (OR 1.68, 95% CI: 1.17–2.40; P = 0.004) and the chance of developing T2DM. Age (rural: OR 1.03, 95% CI: 1.01–1.04; P<0.001; urban: OR 1.02, 95% CI: 1.01–1.04; P<0.001), blood pressure (rural: OR 3.14, 95% CI: 2.0–4.93; P<0.001; urban: OR 2.23, 95% CI: 1.66–3; P<0.001), and abdominal obesity (rural: OR 2.34, 95% CI: 1.41–3.87; P = 0.001; urban: OR 1.46, 95% CI: 1.06–2.01; P = 0.019), in both rural and urban areas, blood cholesterol (OR 1.59, 95% CI: 1.07–2.37; P = 0.02) in rural areas, and blood triglycerides (OR 1.51, 95% CI: 1.16–1.98; P = 0.002) in urban areas were significant predictors of T2DM. Conclusion Given the higher prevalence of T2DM among females, risk reduction strategies at the community level should be more targeted at women. The higher prevalence of T2DM risk factors among the urban population is a wake-up call for policymakers to pay more attention to the consequences of unhealthy and sedentary lifestyles within urban communities. Future actions should be focused on appropriate timely action plans for the prevention and control of T2DM from early years of life.