The work aimed to build the best possible machine learning model predicting the concentration of selected air pollutants and evaluate model accuracy on the days with fires in a station vicinity. The underestimation of a pollutant concentration that coincides with fire would indicate its impact on air quality. Over 1353 thousand cases of fires in Poland and data from 410 air quality stations were analyzed (from 2012-2021). Models for prediction of NO2, NOx, PM10, SO2, and BTEX (benzene, toluene, ethylbenzene, m,p-xylene, and o-xylene) concentrations were built for the carefully selected station (rural background; Borówiec). The accuracy of models was checked as a function of distance from the fire and validated with the dispersion of plumes emitted during big landfill fires in 2018. The share of underpredicted concentrations of PM10, benzene, toluene, and ethylbenzene on days when fire appeared in a range 30 km from the air quality station was significantly higher than the model performance. The concentrations of PM10, SO2, and BTEX, during plumes from landfill fires, were underestimated at least 30% of the duration of exposure. Hence, it was shown that it is very likely fires' contribution to air pollution can be evaluated using machine learning model misclassification.