Recent studies reported different ages for peak freestyle swimming performances for 50 m and 1,500 m. The aims of the present study were (i) to determine the age of peak freestyle swimming speed for distances including 50 m, 100 m, 200 m, 400 m, 800 m, and 1,500 m and to (ii) analyze the sex difference in peak freestyle swimming speed for all distances between 50 m and 1,500 m for elite female and male swimmers competing at national level. Data from the ‘Swiss Swimming Federation’ between 2006 and 2010 from 10,405 men and 9,552 women were analyzed using regression analyses and analyses of variance (ANOVA). Women achieved peak freestyle swimming speed at ~20–21 years from 50 m to 400 m, at ~24–25 years in 1,500 m and at ~25–27 years in 800 m. In men, the age of peak freestyle swimming speed varied between ~22–23 years and ~25–27 years for 50 m to 1,500 m. Between the age of 10 and 29 years, the sex difference in freestyle swimming speed increased from 2.2 ± 0.4% to 19.0 ± 6.7% in 50 m (r2 = 0.87, P < 0.001), from 2.4 ± 0.7% to 10.8 ± 2.8% in 100 m (r2 = 0.67, P = 0.004) and from 3.6 ± 1.9% to 10.2 ± 3.4% in 200 m (r2 = 0.60, P = 0.008). In 400 m (r2 = 0.24), 800 m (r2 = 0.39) and 1,500 m (r2 = 0.34), the sex difference showed no changes (P > 0.05) with 6.9 ± 3.0%, 5.8 ± 3.5%, and 9.7 ± 8.6%, respectively. The sex difference in freestyle swimming speed showed no change with increasing race distance (r2 = 0.12, P > 0.05). To summarize, the age of peak freestyle swimming speed increased for women with the length of the race distance from 50 m to 200 m, but not from 400 m to 1,500 m. For men, the age of peak freestyle swimming speed varied between ~22–23 years and ~25–27 years from 50 m to 1,500 m. The sex difference in freestyle swimming speed of 9.1 ± 2.5% showed no change with increasing race distance. Future studies need to confirm these findings in elite swimmers competing at international level such as the World Championships and the Olympic Games.