This study evaluated the effectiveness of soil and water conservation structures for soil erosion control by applying a semi-distributed Soil and Water Assessment Tool (SWAT) model in various small watersheds of the Chakwal and Attock districts of Pothwar, Pakistan. The validated model without soil conservation structures was applied to various ungauged small watershed sites with soil conservation stone structures. The stone bund-type structure intervention was used in the model through the modification of the Universal Soil Loss Equation (USLE) to support the practice factor (P-factor), the curve number, and the average slope length for the sub-basin (SLSUBBSN). The structures had significant effects, and the average sediment yield reduction caused by the soil conservation stone structures at these sites varied from 40% to 90%. The sediment yield and erosion reductions were also compared under conditions involving vegetation cover change. Agricultural land with winter wheat crops had a higher sediment yield than fallow land with crop residue. The fallow land facilitated sediment yield reduction, along with soil conservation structures. The slope classification analysis indicated that 60% of the agricultural area of the Chakwal and Attock districts lie in a slope range of 0–4%, where considerable potential exists for implementing soil conservation measures by installing soil conservation stone structures. The slope analysis measured the suitability of conservation structures in the semi-mountainous Pothwar area in accordance with agriculture practice on land having a slope of less than 5%. The SWAT model provides reliable performance for erosion control and watershed management in soil erosion-prone areas with steep slopes and heavy rainfall. These findings can serve as references for policymakers and planners.