Recently, the interest in industrial by-products produced at the local level in Mediterranean areas, resulting from fruit and vegetable processes, has increased because of their considerable amounts of bioactive compounds, including polyphenols. In this review, we analyze the most recent scientific results concerning the use of agro-industrial by-products, naturally rich in polyphenols (BPRP), in the diets of small dairy ruminants. Effects on milk production, milk and rumen liquor fatty acid profile, metabolic parameters, and methane production are reviewed. The feed intake and digestibility coefficients were generally depressed by BPRP, even though they were not always reflected in the milk yield. The main observed positive effects of BPRP were on quality of the milk’s FA profile, antioxidant activity in milk and blood, a reduction of rumen ammonia, and, consequently, a reduction of milk and blood urea. The expected beneficial effects of dietary polyphenols in small ruminants were not always observed because of their complex and variable matrices. However, owing to the large quantities of these products available at low prices, the use of BPRB in small ruminant nutrition offers a convenient solution to the valorization of residues arising from agricultural activities, reducing feed costs for farmers and conferring added value to dairy products at the local level, in a sustainable way.