As surface-enhanced Raman scattering (SERS) has been used to diagnose several respiratory viruses (e.g., influenza A virus subtypes such as H1N1 and the new coronavirus SARS-CoV-2), SERS is gaining popularity as a method for diagnosing viruses at the point-of-care. Although the prior and quick diagnosis of respiratory viruses is critical in the outbreak of infectious disease, ELISA, PCR, and RT-PCR have been used to detect respiratory viruses for pandemic control that are limited for point-of-care testing. SERS provides quantitative data with high specificity and sensitivity in a real-time, label-free, and multiplex manner recognizing molecular fingerprints. Recently, the design of Raman spectroscopy system was simplified from a complicated design to a small and easily accessible form that enables point-of-care testing. We review the optical design (e.g., laser wavelength/power and detectors) of commercialized and customized handheld Raman instruments. As respiratory viruses have prominent risk on the pandemic, we review the applications of handheld Raman devices for detecting respiratory viruses. By instrumentation and commercialization advancements, the advent of the portable SERS device creates a fast, accurate, practical, and cost-effective analytical method for virus detection, and would continue to attract more attention in point-of-care testing.