The rapid, low cost, and efficient detection of SARS-CoV-2 virus infection, especially in clinical samples, remains a major challenge. A promising solution to this problem is the combination of a spectroscopic technique: surface-enhanced Raman spectroscopy (SERS) with advanced chemometrics based on machine learning (ML) algorithms. In the present study, we conducted SERS investigations of saliva and nasopharyngeal swabs taken from a cohort of patients (saliva: 175; nasopharyngeal swabs: 114). Obtained SERS spectra were analyzed using a range of classifiers in which random forest (RF) achieved the best results, e.g., for saliva, the precision and recall equals 94.0% and 88.9%, respectively. The results demonstrate that even with a relatively small number of clinical samples, the combination of SERS and shallow machine learning can be used to identify SARS-CoV-2 virus in clinical practice.