Hesitancy about receiving vaccines has been deemed a global danger to public health by WHO. The sociocultural backgrounds of the people have an impact on vaccine acceptance. The purpose of this study was to examine the effect of sociodemographic factors on COVID-19 vaccination hesitancy as well as to identify the factors that contributed to COVID-19 vaccine hesitancy. Methods: A cross-sectional study was carried out to determine the primary variables causing COVID-19 vaccination hesitancy among residents of Pune. The general population was sampled through simple random sampling. The minimum sample size was determined to be 1246. The questionnaire inquired about the individuals’ sociodemographic information, vaccination status, and reasons for vaccine hesitancy. Results: In total, there were 5381 subjects, 1669 of whom were unvaccinated and 3712 of whom were partially vaccinated. Fear of adverse effects (51.71%), fear of losing a few days of work (43.02%), and inability to secure a vaccine slot online (33.01%) were the most frequently cited reasons. An older population (>60 years, p = 0.004), males (p = 0.032), those who were literate (p = 0.011), those of lower middle socioeconomic status (p = 0.001), and smokers were significantly associated with fear and mistrust of the COVID-19 vaccine, while mistrust of the vaccine was greatest among individuals from the upper and lower middle classes (p = 0.001). Conclusion: Vaccine hesitancy due to concerns about the side effects and long-term complications was prevalent among the elderly, males, those from the lower middle class, and smokers. This study emphasizes the importance of communicating effectively about the vaccine’s efficacy, its distribution, and vaccination sites.