Ambient air pollution is a major environmental threat to human health. The acute effects of exposure to ambient air pollution during physical exercise may depend on allergy status. The aim of the study was to assess the acute respiratory responses to air pollution exposure during physical training in young adults with and without allergies. The studied group included 71 healthy young adults (n = 16 with allergy and n = 55 without allergy). Students completed two indoor physical training trials lasting 45–60 min: when air pollutants concentrations were high (exposure trial) and low (control trial). During each trial, we monitored outdoor and indoor environmental conditions. Participants performed spirometry at baseline and directly after the exercise. Exercise during exposure trials led to a small decrease in the percentage of predicted forced expiratory volume in 1 s (FEV1 ref). Only during the control trials did the FEV1/forced vital capacity quotient (FEV1/FVC) statistically significantly increase. Moreover, just in the allergy group, there were statistically significant negative correlations between post-exercise FEV1/FVC change and 3 h average outdoor particulate matter with aerodynamic diameter <10 µm (PM10) and nitrogen dioxide (NO2) concentrations (PM10: r = −0.54, p = 0.02, NO2: r = −0.60, p = 0.02). In young and healthy adults, sports training under exposure to high levels of ambient air pollutants leads to a small decrease in FEV1. The allergy might be a modifying factor in the respiratory responses to air pollution. Post-exercise decrease in FEV1/FVC was related to pre-exercise 3 h averages of PM10 and NO2 only in people with ever-diagnosed upper-respiratory allergy.