Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

Abstract

Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well-defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanisms is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training on a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.

Unsupervised microstructure segmentation by mimicking metallurgists’ approach to pattern recognition

“…2 Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-8656, Japan. * email: inoue@ material.t.u-tokyo.ac.jp www.nature.com/scientificreports/ to detect defects in steels 20 and ResNet18 21 to classify microstructures of welded steels 22 . It was verified that the performance of CNN-based methods is as good as that of humans.…”

Automatic monitoring of steel strip positioning error based on semantic segmentation

“…Semantic segmentation is a pixel-wise categorization, which gathers pixels belonging to the same class [24,28,30]. Regarding digital image processing, the method is best applied as an emulator for human pattern identification [28].…”

“…Semantic segmentation is a pixel-wise categorization, which gathers pixels belonging to the same class [24,28,30]. Regarding digital image processing, the method is best applied as an emulator for human pattern identification [28]. Compared to the traditional image segmentation, semantic segmentation based on convolution neural network has demonstrated considerable advantages [28] and has been applied to many tasks such as medical applications [2, 26,27,42], in autonomous driving [6,8,12,32], object detection [13,37], and pose estimation system [33], to name a few.…”

“…The available literature presents a considerable number of studies with solutions for metalworking. These include the detection of defects in metal casting [5,10], recognition of slab identification [22], prediction of mechanical properties [38], bearing fault diagnosis [14,19,29,35,41], and steel defect identification [25,28,31]. On the other hand, there are only a few works that apply image processing to measure the strip position in a rolling mill process.…”

Automated Quantitative Analysis of Extended Irradiation Defects - Dislocations, Voids and Precipitates in Neutron Irradiated HT-9 Steel

“…To mitigate this problem, as shown in Figure 2d, we modified the popular circular Hough transformation and treated 'circular shape' as the particle descriptor for voids/precipitates, efficiently resolving the overlapping particles. In summary, this automated digital quantification together with machine learning enabled defect identification [3] opens new opportunities for standardizing the characterization workflow for quantitative analysis of extended irradiation defects. Figure 1.…”