Deep learning-based semantic segmentation for in-process monitoring in laser welding applications

Knaak, Christian; Kolter, Gerald; Schulze, Frederic; Kröger, Moritz; Abels, Peter

doi:10.1117/12.2529160

Cited by 23 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The test result of regression is evaluated with the coefficient of determination

[ 5 ]:

where

is the number of samples,

is the predicted value of the

-th sample,

is the corresponding true value,

is the mean value of

of all samples. The best possible value of

is 1.0.…”

Section: Architecture Of the Modelmentioning

confidence: 99%

“…Since multiple factors influence the appearance of the melt pool, images of the melt pool captured by a vision sensor contain rich information about the welding process [ 5 ], and can reflect the condition of the welding process, namely welding state. In addition, compared with other sensors like an acoustic emission sensor and photodiode sensor, a vision sensor has advantages considering the quality of the collected information, the position of the sensor and the industrial application.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, researchers frequently used vision sensors to capture images of the welding process and monitored the welding state [ 6 ]. Hand-crafted image processing algorithms were used to extract features from images and machine learning methods like support vector machine [ 7 , 8 , 9 , 10 ], random forest [ 5 ], and k-nearest neighbors [ 11 ] were applied to predict the welding state. However, image data captured by vision sensors can be complex and high-dimensional.…”

Section: Introductionmentioning

confidence: 99%

“…where a h and a nh are classification accuracies on samples with ground-truth labels of humping and no humping, respectively. The test result of regression is evaluated with the coefficient of determination R 2 [5]:…”

mentioning

confidence: 99%

See 3 more Smart Citations

Multi-Output Monitoring of High-Speed Laser Welding State Based on Deep Learning

Xue

Chang

2021

Sensors

View full text Add to dashboard Cite

In order to ensure the production quality of high-speed laser welding, it is necessary to simultaneously monitor multiple state properties. Monitoring methods combining vision sensing and deep learning models are popular but most models used can only make predictions on single welding state property. In this contribution, we propose a multi-output model based on a lightweight convolutional neural network (CNN) architecture and introduce the particle swarm optimization (PSO) technique to optimize the loss function of the model, to simultaneously monitor multiple state properties of high-speed laser welding of AISI 304 austenitic stainless steel. High-speed imaging is performed to capture images of the melt pool and the dataset is built. Test results of different models show that the proposed model can achieve monitoring of multiple welding state properties accurately and efficiently. In addition, we make an interpretation and discussion on the prediction of the model through a visualization method, which can help to deepen our understanding of the relationship between the melt pool appearance and welding state. The proposed method can not only be applied to the monitoring of high-speed laser welding but also has the potential to be used in other procedures of welding state monitoring.

show abstract

“…The test result of regression is evaluated with the coefficient of determination

[ 5 ]:

where

is the number of samples,

is the predicted value of the

-th sample,

is the corresponding true value,

is the mean value of

of all samples. The best possible value of

is 1.0.…”

Section: Architecture Of the Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Multi-Output Monitoring of High-Speed Laser Welding State Based on Deep Learning

Xue

Chang

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The keyhole is typically surrounded by molten material, the weld pool. Size and shape of weld pool are important geometrical parameters that correlate with weld shape and quality [ 10 , 11 ].…”

Section: Introductionmentioning

confidence: 99%

A Spatio-Temporal Ensemble Deep Learning Architecture for Real-Time Defect Detection during Laser Welding on Low Power Embedded Computing Boards

Knaak

Eßen

Kröger

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

In modern production environments, advanced and intelligent process monitoring strategies are required to enable an unambiguous diagnosis of the process situation and thus of the final component quality. In addition, the ability to recognize the current state of product quality in real-time is an important prerequisite for autonomous and self-improving manufacturing systems. To address these needs, this study investigates a novel ensemble deep learning architecture based on convolutional neural networks (CNN), gated recurrent units (GRU) combined with high-performance classification algorithms such as k-nearest neighbors (kNN) and support vector machines (SVM). The architecture uses spatio-temporal features extracted from infrared image sequences to locate critical welding defects including lack of fusion (false friends), sagging, lack of penetration, and geometric deviations of the weld seam. In order to evaluate the proposed architecture, this study investigates a comprehensive scheme based on classical machine learning methods using manual feature extraction and state-of-the-art deep learning algorithms. Optimal hyperparameters for each algorithm are determined by an extensive grid search. Additional work is conducted to investigate the significance of various geometrical, statistical and spatio-temporal features extracted from the keyhole and weld pool regions. The proposed method is finally validated on previously unknown welding trials, achieving the highest detection rates and the most robust weld defect recognition among all classification methods investigated in this work. Ultimately, the ensemble deep neural network is implemented and optimized to operate on low-power embedded computing devices with low latency (1.1 ms), demonstrating sufficient performance for real-time applications.

show abstract

Optimisation of laser welding of deep drawing steel for automotive applications by Machine Learning: A comparison of different techniques

Maculotti

Genta

Galetto

2023

Quality & Reliability Eng

View full text Add to dashboard Cite

Laser welding is particularly relevant in the industry thanks to its simplicity, flexibility and final quality. The industry 4.0 and sustainable manufacturing framework gives massive attention to in situ and non‐destructive inspection methods to predict laser weld final quality. Literature often resorts to supervised Machine Learning approaches. However, selecting the ApTest method is non‐trivial and often decision making relies on diverse and unclearly defined criteria. This work addresses this task by proposing a statistical comparison method based on nonparametric tests. The method is applied to the most relevant supervised Machine Learning approaches exploited in literature to predict laser weld quality, specifically, considering the optimisation of a new production line, hence focussing on supervised Machine Learning methods that do not require massive data set, that is, Generalized Linear Model (GLM), Gaussian Process Regression, Support Vector Machine, Classification and Regression Tree, and Genetic Algorithms. The statistical comparison is carried out to select the best‐performing model, which is then exploited to optimise the production process. Additionally, an automatic process to optimise Machine Learning models and process parameters is resorted to, basing on Bayesian approaches, to reduce operator effect. This work provides quality and process engineers with a simple framework to compare Machine Learning approaches performances and select the most suitable process modelling technique.

show abstract

Deep learning-based semantic segmentation for in-process monitoring in laser welding applications

Cited by 23 publications

References 15 publications

Multi-Output Monitoring of High-Speed Laser Welding State Based on Deep Learning

Multi-Output Monitoring of High-Speed Laser Welding State Based on Deep Learning

A Spatio-Temporal Ensemble Deep Learning Architecture for Real-Time Defect Detection during Laser Welding on Low Power Embedded Computing Boards

Optimisation of laser welding of deep drawing steel for automotive applications by Machine Learning: A comparison of different techniques

Contact Info

Product

Resources

About