Background The objective of this study was to identify clinical risk factors for COVID-19 in a German outpatient fever clinic that allow distinction of SARS-CoV-2 infected patients from other patients with flu-like symptoms. Methods This is a retrospective, single-centre cohort study. Patients were included visiting the fever clinic from 4th of April 2020 to 15th of May 2020. Symptoms, comorbidities, and socio-demographic factors were recorded in a standardized fashion. Multivariate logistic regression was used to identify risk factors of COVID-19, on the bases of those a model discrimination was assessed using area under the receiver operation curves (AUROC). Results The final analysis included 930 patients, of which 74 (8%) had COVID-19. Anosmia (OR 10.71; CI 6.07–18.9) and ageusia (OR 9.3; CI 5.36–16.12) were strongly associated with COVID-19. High-risk exposure (OR 12.20; CI 6.80–21.90), especially in the same household (OR 4.14; CI 1.28–13.33), was also correlated; the more household members, especially with flu-like symptoms, the higher the risk of COVID-19. Working in an essential workplace was also associated with COVID-19 (OR 2.35; CI 1.40–3.96), whereas smoking was inversely correlated (OR 0.19; CI 0.08–0.44). A model that considered risk factors like anosmia, ageusia, concomitant of symptomatic household members and smoking well discriminated COVID-19 patients from other patients with flu-like symptoms (AUROC 0.84). Conclusions We report a set of four readily available clinical parameters that allow the identification of high-risk individuals of COVID-19. Our study will not replace molecular testing but will help guide containment efforts while waiting for test results.