The processing of agricultural products into value-added food products yields numerous by-products or waste streams such as pomace (fruit and vegetable processing), hull/bran (grain milling), meal/cake (oil extraction), bagasse (sugar processing), brewer's spent grain (brewing), cottonseed meal (cotton processing), among others. In the past, significant work in exploring the possibility of the utilization of these by-products has been performed. Most by-products are highly nutritious and can be excellent low-cost sources of dietary fiber, proteins, and bioactive compounds such as polyphenols, antioxidants, and vitamins. The amount of energy utilized for the disposal of these materials is far less than the energy required for the purification of these materials for valorization. Thus, in many cases, these materials go to waste or landfill. Studies have been conducted to incorporate the by-products into different foods in order to promote their utilization and tackle their environmental impacts. Extrusion processing can be an excellent avenue for the utilization of these by-products in foods. Extrusion is a widely used thermo-mechanical process due to its versatility, flexibility, high production rate, low cost, and energy efficiency. Extruded products such as direct-expanded products, breakfast cereals, and pasta have been developed by researchers using agricultural by-products. The different by-products have a wide range of characteristics in terms of chemical composition and functional properties, affecting the final products in extrusion processing. For the practical applications of these by-products in extrusion, it is crucial to understand their impacts on the qualities of raw material blends and extruded products. This review summarizes the general differences in the properties of food by-products from different sources (proximate compositions, physicochemical properties, and functional properties) and how these properties and the extrusion processing conditions influence the product characteristics. The discussion of the by-product properties and their impacts on the extrudates and their nutritional profile can be useful for food manufacturers and researchers to expand their applications. The gaps in the literature have been highlighted for further research and better utilization of by-products with extrusion processing.