During the coronavirus disease 2019 (COVID-19) pandemic, several restriction measures were imposed to control the virus transmission, with important repercussions on different sectors, including sport. This study aimed to explore the effects of the COVID-19 pandemic on Italian competitive swimmers by analyzing how the disease and the restriction measures affected their training. In total, 396 competitive swimmers (mean age 16.0 ± 3.2 years) participated. A questionnaire was used to collect their general information, to assess whether they had had COVID-19 and the number of training days lost due to the disease or to the closure of swimming facilities, and the possible alternative training adopted. Twenty-four (6.1%) participants had had COVID-19 and lost, on average, 32 training days. The closure of facilities caused an interruption in swimming training for about 18% of the participants. The majority of these continued their training, mainly through home-based exercise, but reduced their weekly training time (-8 median hours/week). A positive association was found between regularly adopted weekly training volume and that assumed during pandemic closure (OR 9.433, CI95% 1.644–54.137, p = 0.012), suggesting that the previous level of engagement in sport can represent a predictor of exercise maintenance in challenging situations such as a pandemic. Further studies are needed to identify personal, environmental, and social resources that can help individuals to counteract the negative effects of restriction measures.