The objective of this study was to investigate the incidence and course of COVID-19 and the risk of an upper respiratory tract infection in a group of people with physically active lifestyles. Data were collected anonymously using an online survey platform during December 2020. The age of participants ranged from 18 to 65 years. Out of 2343 participants, 11.5% overcame COVID-19 infection. Relative to the control group (CTRL), physically active, cold-water swimmers (PACW) did not exhibit a lower risk of incidence for COVID-19 (RR 1.074, CI 95% (0.710–1.625). However, PACW had a higher chance of having an asymptomatic course of COVID-19 (RR 2.321, CI 95% (0.836–6.442); p < 0.05) and a higher chance of only having an acute respiratory infection once or less per year than CTRL (RR 1.923, CI 95% (1.1641–2.253); p < 0.01). Furthermore, PACW exhibited a lower incidence of acute respiratory infection occurring more than twice per year (RR 0.258, CI 95% (0.138–0.483); p < 0.01). Cold-water swimming and physical activity may not lessen the risk of COVID-19 in recreational athletes. However, a physically active lifestyle might have a positive effect on the rate of incidence of acute respiratory infection and on the severity of COVID-19 symptoms.