This study was conducted in the Atwood and Tappan Lakes watersheds of the Tuscarawas basin of Ohio. The flow, total nitrogen (TN), and total phosphorus (TP) loadings were monitored with the help of local stakeholders for a few years at various locations of the watershed to develop the Soil and Water Assessment Tool (SWAT). The multi-site SWAT model calibration and validation were accomplished with a reasonable model performance. In the next step, the scenario analysis was conducted in the SWAT model using various BMPs, including vegetative filter strips, grass waterways, fertilizer reduction, crop rotation, and cover crops to evaluate their performance in reducing TN and TP from the watershed. While BMPS in many studies are decided based on researchers’ intuition, these BMPs were selected based on active consultation with the local stakeholders, who were engaged in the reduction of TN and TP loadings from the watersheds. Since the SWAT model calibration for TN and TP was not as good as the hydrologic model calibration, various scenarios of TN and TP reduction using BMPs were investigated for several years using both calibrated and uncalibrated SWAT models. We examined all the BMPs in 12 sub-watersheds of the Atwood and 10 sub-watersheds of the Tappan Lake watershed. The analysis indicated that the management practices of cover crops (rye) in combination with grass waterways with a 10% fertilizer reduction could minimize the TN and TP loading by as much as 88%, without significantly compromising the agricultural yield. However, a 10% fertilizer reduction without any BMPs could reduce TN and TP by just 9%. The cover crop (rye) including 10% fertilizer reduction with grass waterways seemed to be the most effective in reducing TN and TP, whereas the implementation of a filter strip led to a 70% reduction and was the next effective BMPs in reducing TN and TP loadings. In general, TN losses were reduced by 8% to 53%, while TP losses were reduced by 7% to 88%, depending on the BMPs used. By and large, the TN and TP reduction achieved through the calibrated model was not significantly different from the uncalibrated model, even though the reduction using the calibrated model was slightly higher for all scenarios than that of the uncalibrated model. The TN and TP loadings were highly sensitive to cattle grazing. When just 50% of the cattle were permitted to graze, the model predicted that there would be a 40% increase in total nitrogen and a 70% increase in total phosphorus in both watersheds. Our investigation revealed that monitoring the watershed at a small sub-watershed scale and calibrating the SWAT model for nitrogen and phosphorus is delicate.