Computer vision and deep learning techniques are the most emerging technologies in this era. Both of these can greatly raise the rate at which defects on metal surfaces are identified while performing industrial quality checks. The identification of faults over metal surfaces can be viewed as a significant challenge since they are easily impacted by ambient factors including illumination and light reflections. This paper proposes novel metal surface defect detection network called as YOLOv-5s-FRN in response to the problems of ineffective detection brought by the conventional manual inspection system. The proposed system is developed through the integration of a novel architectural module called as Feature Recalibration Network (FRN) to the You Only Look Once-version-5 small network )YOLOv-5s(. In order to extract the global feature information from the provided image, FRN is able to evaluate the interdependencies between the channels. This improves the feature discrimination capability and prediction accuracy of the defect detection system. The incorporation of FRN structure makes YOLOv-5s architecture to selectively enhance the necessary features and discard the unwanted ones. Therefore, the proposed novel method will efficiently detect and classify the metal surface defects such as crazing, patches, inclusions, scratches, pitted surfaces and rolled in scale. North Eastern University Surface Defect Database (NEU-DET) has been used to train and test the proposed architectural model. The suggested system has been compared with alternative models based on several performance matrices such as precision, recall and Mean Average Precision (mAP). It is observed that the proposed YOLOv-5s-FRN architecture provides significant performance improvement than state-of-the-art methods. The proposed system has been provided satisfactory results by means of improvement in mAP and time consumption. The proposed model has delivered value of mAP_0.5 as 98.05% and that of mAP_0.5:0.95 as 89.03%.