Due to the complex scenes of reality in the scrapyard, the overlapping between scrap metals causes occlusion, resulting in missed detection and false detection problems during the classification and grading of scrap metals. In response to this problem, we propose a network model for scrap metal classification and grading, named FG-Net, which improves Yolov5s. First, we replace the convolutional layers in the backbone with the proposed MBCC module, so that the backbone network can adopt strategies such as depth-wise separable convolution, residual connection, channel and spatial expansion, which not only improves the feature extraction ability, but also enhances the efficiency and accuracy of the network. In the feature fusion stage of this model, we introduce the RFB module to improve the receptive field and increase the flexibility of feature fusion, enabling the model to fuse more feature information. Finally, an improved loss function is used to address the missed detection and false detection issues caused by the closeness of scrap metals.This paper first verifies the model on the official datasets PASCAL VOC(2007+2012) and COCO, with mAPs of 84.6% and 59.8%, respectively, which are 2.1% and 3% higher than Yolov5s. The accuracy of the self-made scrap metal dataset reaches 87.4%, with an mAP of 86.1%, which is 10.1% and 10.4% higher than Yolov5s, respectively. Through experiments, it is proved that this model not only improves the accuracy of scrap metal recognition, but also solves the occlusion problem in the current scrap metal classification and grading process.