Serological testing is a powerful tool for analyzing the infectious disease burden landscape. Therefore, this study aimed to determine the seroprevalence against SARS-CoV-2 in the population of the municipality of Kragujevac, Serbia, with a particular reference to silent infections. A total of 4,804 participants over 19 years of age were randomly sampled for population-based seroprevalence research. Anti-N IgG antibodies were measured using rapid serological tests (UNscience®). The population was divided into four Cohorts, according to the history of SARS-CoV-2 infection and vaccination status with the whole inactivated virus vaccine BBIBP-CorV (Vero Cell®, Sinopharm), as follows: Cohort I—confirmed SARS-CoV-2 infection, not vaccinated with the BBIBP-CorV vaccine; Cohort II—without confirmed SARS- CoV-2 infection, vaccinated with the BBIBP-CorV vaccine; Cohort III—confirmed SARS-CoV-2 infection, vaccinated with the BBIBP-CorV vaccine; Cohort IV—without confirmed SARS-CoV-2 infection, not vaccinated with the BBIBP-CorV vaccine (silent immunization). Cohorts I and IV included patients vaccinated with vaccines other than the BBIBP-CorV vaccine. The results showed that the overall prevalence of anti-N IgG antibodies was 56.5%, with the highest seroprevalence in Cohort III at 85.8%. In Cohort IV, the prevalence of anti-N IgG antibodies was 40.7%, attributed to silent immunization. The results also suggest that the prevalence of anti-N IgG antibodies decreased over time but remained detectable for more than 12 months in Cohort I. Since currently, there is no data on silent infection frequency in our country, these findings may provide insight into the extent of silent infections in the Serbian population.