The 2019 Coronavirus (COVID-19) outbreak caused home confinement, as well as training and sport competitions withdrawals. The prolonged inactivity impact, and lack of in-person interactions among teammates-coaches, could negatively affect athletes. Total of 1508 self-selected Italian athletes, 338 children (aged 10.52 ± 1.31), 499 adolescents (aged 14.17 ± 1.13), and 671 adults (aged 27.59 ± 10.73), completed the Impact of Event Scale (IES-8, IES-15, and IES-R, respectively). Differences by gender, type of sport (individual vs. team), and competitive level (elite vs. amateur) were examined. One-way ANOVAs showed, in adults, significant differences between genders for perceived stress impact total score (TS; p = 0.017) and avoidance behavior, with higher scores in women (p = 0.045). Between individual and team sport, significant differences were found in TS (p = 0.038) and hyperarousal (p = 0.030), with higher results in individual. Adult elite athletes showed significantly higher scores in hyperarousal (p = 0.020) than amateurs. Significant differences were found between gender in adolescents for avoidance (p = 0.011), and between competitive levels in children, for intrusion (p = 0.020). These evidences may raise awareness on distress effects of COVID-19 lockdown among athletes and suggested applying specific well-being protocols during the activity resumption, considering gender, type of sport, and competitive level.