Excessive phosphorus is an important cause of eutrophication. For river basin management, source identification and control of nonpoint source (NPS) pollution are difficult. In this study, to explore influences of hydrological conditions on phosphorus, the Soil and Water Assessment Tool (SWAT) model is applied to the Luanhe River basin in North China. Moreover, influences of the spatial scale of the livestock and poultry amount data on estimations of phosphorus loads are also discussed. The results show that applying town-level livestock and poultry amount data allows the model to perform better when estimating phosphorus loads, indicating that using data at a finer administrative level is necessary. For the typical wet year, the estimated annual phosphorus load was 2.6 times that in the typical dry year. Meanwhile, the contribution of pollution in summer to the annual load is greater in the wet year than that in the dry year. The spatial distributions of subbasins with high unit loads of phosphorus differ under different hydrological conditions, meaning that critical areas for pollution control vary with the wetness of each year. All these findings indicate that for pollution control at basin scale, considering the seasonal and interannual variabilities in hydrological conditions is highly demanded.