Evidence based findings on long-term health-related consequences of a SARS-CoV-2 infection remain scarce. Data from wearable devices, well suited for continuous measurement of heart rate and physical activity, offers a unique opportunity to assess the impact of such infections on an individual's health. Here we aim to characterize comprehensively how persistent self-reported symptoms during both acute and post-acute infection correlate to changes in resting heart rate (RHR) and physical activity, as measured by consumer-grade wearable sensors. Using a wearable-derived dataset of behavior and physiology (n = 20,815), we identified 137 individuals who are characterized by persistent fatigue and shortness of breath after a reported positive SARS-CoV-2 test. We compared this cohort with COVID-19 positive without persistent symptoms and negative controls. The comparison is based on measurements of RHR and physical activity as well as self-reported health-related Quality of Life (QoL) through WHO-5 and EQ-5D before, during, and after the infection. We identified a unique phenotype of persistent COVID-19 symptoms and associated wearable data characteristics and compared this phenotype to COVID-19 positive and negative controls. Individuals who reported persistent symptoms (coexisting shortness of breath and fatigue) showed higher RHRs (mean difference of 2.37/1.49 bpm), and lower daily step count (on average 3,030/2,909 steps less) compared to positive/negative controls, even at least three weeks prior to a SARS-CoV-2 infection. During the acute phase (0-4 weeks after a positive COVID-19 test), individuals with persistent shortness of breath and fatigue exhibited a decrease in mean RHR, 1.86 times that of individuals in the positive control cohort. Similarly, the persistent symptom phenotype took an average of seven days longer to return to normal compared to positive controls. Additionally we found that self-reported persistent COVID-19 symptoms are linked to a substantial reduction in mean QoL, even before infection. The analysis of individual wearable time-series suggests that the persistent symptom phenotype, characterized by shortness of breath and fatigue, may have been more exposed to pre-existing health conditions and/or exhibited lower levels of fitness prior to a SARS-CoV-2 infection. Our approach demonstrates the enormous potential in tracking the dynamics of physiological and physical activity under natural conditions in the context of infectious and chronic diseases.