Cell-intrinsic responses mounted in vivo in PBMCs during mild and severe COVID-19 differ quantitatively and qualitatively. Whether they are triggered by signals emitted by productively infected cells of the respiratory tract or are, at least partially, resulting from physical interaction with virus particles, remains unclear. Here, we analyzed susceptibility and expression profiles of PBMCs from healthy donors upon ex vivo exposure to SARS-CoV and SARS-CoV-2. In line with the absence of detectable ACE2 receptor expression, human PBMCs were refractory to productive infection. Bulk and single cell RNA-sequencing revealed JAK/STAT-dependent induction of interferon-stimulated genes, but not pro-inflammatory cytokines. This SARS-CoV-2-specific response was most pronounced in monocytes. SARS-CoV-2-RNA-positive monocytes displayed a lower ISG signature as compared to bystander cells of the identical culture. This suggests a preferential invasion of cells with a low ISG base-line profile or delivery of a SARS-CoV-2-specific sensing antagonist upon efficient particle internalization. Together, non-productive physical interaction of PBMCs with SARS-CoV-2- but not SARS-CoV particles stimulates JAK/STAT-dependent, monocyte-accentuated innate immune responses that resemble those detected in vivo in patients with mild COVID-19.