Accumulating interest from academia and industry, the part of quality assurance in metal additive manufacturing (AM) is achieving incremental recognition owing to its distinct advantages over conventional manufacturing methods. In this paper, we introduced a convolutional neural network, YOLOv8 approach toward robust metallographic image quality inspection. Metallographic images accommodate key information relating to metal properties, such as structural strength, ductility, toughness, and defects, which are employed to select suitable materials for multiple engineering execution. Therefore, by comprehending the microstructures, one can understand insights into the behavior of a metal component and make predictive assessments of failure under specific conditions. Deep learning-based image segmentation is a robust technique for the detection of microstructural defects like cracks, inclusion, and gas porosity. Therefore, we improvise the YOLOv8 with dilated convolution mechanisms to acquire automatic micro-structure defect characterization. More specifically, for the first time, the YOLOv8 algorithm was proposed in the metallography dataset from additive manufacturing of steels (Metal DAM) to identify defects like cracks and porosity as a novel approach. A total of 414 images from ArcelorMittal engineers were used as an open-access database. The experimental results demonstrated that the YOLOv8 model successfully detected and identified cracks and porosity in the metal AM dataset, achieving an improved defect detection accuracy of up to 96% within just 0.5 h compared to previous automatic defect recognition processes.