The accurate detection of SARS-CoV-2 through respiratory sampling is critical for the prevention of further transmission and timely initiation of treatment. There is a diverse range of SARS-CoV-2 detection rates in reported studies, with uncertainty regarding the optimal sampling method for COVID-19 diagnosis and monitoring. Oropharyngeal sampling (OPS) is one of the most commonly used methods of respiratory sampling in Ghana and other parts of the world for the detection of SARS-CoV-2 viral RNA. However, this sampling technique has a number of drawbacks, which include difficulty in obtaining high-quality swab samples, increased risk of infection to healthcare workers, and increased cost from a regular supply of swabs, transport media, and personal protective equipment (PPE). This study, therefore, sought to evaluate the diagnostic performance of sputum specimens in the diagnosis of COVID-19. This was a cross-sectional analytical study conducted in two health facilities in Kumasi, Ghana, between April and September 2021. Paired samples (an oropharyngeal swab and sputum) were taken from each recruited patient and run concurrently for the detection of SARS-CoV-2 genes (the N and ORF1ab genes) using RT-qPCR. Of the 317 patients recruited, 50.8% were males, and 60.4% were young adults aged 20–39 years. A significant proportion (65.9%) of the patients did not have any co-morbidity, and the majority were with symptoms; predominantly cough (36.3%), headache (31.5%), general weakness (24.0%), fever (20.2%), and sore throat (16.1%). Being symptomatic (p = 0.003), having comorbidity (p = 0.001), and the reporting facility (p = 0.010) were significantly associated with the COVID-19 status. The sputum samples yielded more COVID-positive, 120/317 (37.9%), as compared to OPS, 83/317 (26.2%). The sputum samples were 85.5% (95% CI, 76.4–91.5) sensitive, 79.1% (95% CI, 73.4–83.7) specific, and with positive and negative predictive values of 59.2% and 93.9%, respectively, when compared with OPS. The overall median of the SARS-CoV-2 viral loads for sputum (3.70 × 103 copies/mL) were significantly higher than in OPS (1.18 × 102 copies/mL) (p = 0.003). Findings from the study suggest self-collected sputum as a useful alternative to OPS for the diagnosis of COVID-19, providing a comparable diagnostic performance and, thereby, easing the uncomfortable process and mitigating risk of aerosol transmission to healthcare workers.