Due to decarbonization commitment made by steelmaking companies, the steel industry is tackling a technological transition from blast furnace (BF)–basic oxygen furnace (BOF) route to direct reduction iron (DRI)–electric arc furnace (EAF) route. Under this scenario, ferrous scrap becomes a critical factor for reaching CO2 reduction challenge. However, ferrous scrap can be considered one of the most complex industrial raw materials. In addition, scrap presents a huge heterogeneity in both physical and chemical characteristics. However, for producing high‐quality steel products, certainty on scrap specifics is required. Herein, an artificial intelligent model based on spectral information for the segmentation of different materials contained in the ferrous scrap is proposed. Developed solution offers a processing pipeline through a 2D–3D convolutional neural network algorithm based on a dataset with more than 428 million of pixels through hyperspectral cameras in the 400–1700 nm range. By this model, the detection of ferric fraction, stainless steel, aluminum, zinc, copper, sterile, and rubber and plastic materials are assessed. This work aims at increasing the reliability of the steelmaking process by lowering the number of steel quality noncompliance rejection due to lack of knowledge and uncertainties of these raw material compositions.