(1) Background: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and its resulting coronavirus disease 2019 (COVID-19) has caused a fast-moving pandemic. Diagnostic testing, aimed to identify patients infected with SARS-CoV-2, plays a key role in controlling the COVID-19 pandemic in different populations. (2) Methods: This retrospective cohort study aimed to investigate predictors associated with positive polymerase chain reaction (pcr) SARS-CoV-2 test results in hospitalized patients, healthcare workers (HCWs), and military personnel (MP) during 2020, before the widespread availability of COVID-19 vaccines. Persons with a positive test result were compared with persons with a negative test result in three cohorts during the study period. (3) Results: A total of 6912 respondents were tested, and 1334 (19.3%) of them had positive PCR SARS-CoV-2 test results. Contact with a known COVID-19 case within 14 days (p < 0.001; OR: 1.48; 95% CI: 1.25–1.76), fever (p < 0.001; OR: 3.66; 95% CI: 3.04–4.41), cough (p < 0.001; OR: 1.91; 95%CI: 1.59–2.30), headache (p = 0.028; OR: 1.24; 95%CI: 1.02–1.50), and myalgia/arthralgia (p < 0.001; OR: 1.99; 95%CI: 1.65–2.42) were independently associated with positive PCR SARS-CoV-2 test results in the cohort of MP. Furthermore, fever (p < 0.001; OR: 2.75; 95% CI: 1.83–4.13), cough (p < 0.001; OR: 2.04; 95%CI: 1.32–3.13), headache (p = 0.008; OR: 1.76; 95%CI: 1.15–2.68), and myalgia/arthralgia (p = 0.039; OR: 1.58; 95%CI: 1.02–2.45) were independently associated with positive PCR SARS-CoV-2 test results in the cohort of HCWs. Moreover, independent predictors of positive PCR SARS-CoV-2 test results in hospitalized patients were contact with a known COVID-19 case within 14 days (p < 0.001; OR: 2.56; 95%CI: 1.71–3.83), fever (p < 0.001; OR: 1.89; 95%CI: 1.38–2.59), pneumonia (p = 0.041; OR: 1.45; 95%CI: 1.01–2.09), and neurological diseases (p = 0.009; OR: 0.375; 95%CI: 0.18–0.78). (4) Conclusions: According to data gathered from cohorts of hospitalized patients, HCWs, and MP, before the widespread availability of COVID-19 vaccines in Serbia, we can conclude that predictors of positive PCR SARS-CoV-2 test results in MP and HCWs were similar. Accurate estimates of COVID-19 in different population groups are important for health authorities.