Small abandoned mining areas of Fe and Mn oxides located in the Portuguese sector of the Iberian Pyrite Belt (SW of Europe) have been converted into agrosilvopastoral systems with very few environmental management measures after their closure. Although at the landscape scale, no visible differences were observed between the former mining intervention areas and adjacent areas, it is essential to assess the state and environmental risk of the soil–plant system, especially in the herbaceous pastures grazed by domestic animals. This was carried out in the Ferragudo mining area, where an agrosilvopastoral system, composed of holm oak and dryland pasture, had been established after the closure of the mine at ≈45 years. The soils presented neutral pH and variable fertility degree. The pseudo-total soil concentrations of Cu, Mo, and Zn exceeded the Portuguese limit values established for agriculture use (>180 mg Cu/kg; >8.2 mg Mo/kg; 349 mg Zn/kg), but their soil available fractions were small (<8.4% of the pseudo-total concentrations). Trees and herbaceous plants showed good development, and the concentrations of the elements (except Mn) were considered normal or sufficient. For Mn, most of the plant samples exceeded phytotoxic Mn values, but no visual signs of phytotoxicity were observed. Only the concentrations of Fe and K in the shoots of some herbaceous samples exceeded the maximum tolerable levels for cattle and sheep, so the risk to animals can be considered small since other sources are present in animal feed. In general, this agrosilvopastoral system did not pose a significant environmental risk.