The COVID-19 pandemic can have severe mental implications. In this regard, this study investigates the possibility that resilience and anxiety can mediate the effect of perceived disturbance from the COVID-19 outbreak on psychological well-being. It also examines the moderating effect of a reduction in physical activity on this relationship. To that end, an eligible sample including 863 participants (age range: 18–67 years) completed the 18-item Ryff’s Psychological Well-Being Scale (PWB-18), the Connor–Davidson Resilience Scale, and the COVID-19 Related Anxiety Scale. They also rated two items assessing perceived COVID-19 disturbance and reduction in physical activity during the pandemic. The study confirms that coronavirus anxiety and resilience mediate the predictive relation between perceived COVID-19 disturbance and psychological well-being. Findings also indicate that a reduction in physical activity moderates the effect of perceived disturbance from COVID-19 on psychological well-being, resilience, as well as coronavirus anxiety. Individuals with reduced physical activity due to the COVID-19 pandemic are more affected by the disrupting impact of the pandemic on psychological well-being. This study contributes to the comprehension of factors involved in the disruptive impact of pandemics on psychological well-being.