Improving Automated Visual Fault Detection by Combining a Biologically Plausible Model of Visual Attention with Deep Learning

Abstract

It is a long-term goal to transfer biological processing principles as well as the power of human recognition into machine vision and engineering systems. One of such principles is visual attention, a smart human concept which focuses processing on a part of a scene. In this contribution, we utilize attention to improve the automatic detection of defect patterns for wafers within the domain of semiconductor manufacturing. Previous works in the domain have often utilized classical machine learning approaches such as KNNs, SVMs, or MLPs, while a few have already used modern approaches like deep neural networks (DNNs). However, one problem in the domain is that the faults are often very small and have to be detected within a larger size of the chip or even the wafer. Therefore, small structures in the size of pixels have to be detected in a vast amount of image data. One interesting principle of the human brain for solving this problem is visual attention. Hence, we employ here a biologically plausible model of visual attention for automatic visual inspection. On this basis, we propose a hybrid system of visual attention and a deep neural network. As demonstrated, our system achieves among other decisive advantages an improvement in accuracy from 81 % to 92 %, and an increase in accuracy for detecting faults from 67 % to 88 %. Therefore, the error rates are reduced from 19 % to 8 %, and notably from 33 % to 12 % for detecting a fault in a chip. Hence, these results show that attention can greatly improve the performance of visual inspection systems. Furthermore, we conduct a broad evaluation, which identifies specific advantages of the biological attention model in this application, and benchmarks standard deep learning approaches as an alternative with and without attention. This work is an extended arXiv version of the original conference article published in "IECON 2020". It has been extended regarding visual attention, covering (i) the improvement and equations of visual attention model, (ii) a deeper evaluation of the model, (iii) a discussion about possibilities to combine the attention model with the DNN, and (iv) a detailed overview about the data.

Improving automated visual fault inspection for semiconductor manufacturing using a hybrid multistage system of deep neural networks

“…The data set differs from our previous contributions due to a reassessment of the data, leading as a notable difference to a revised defect pattern class specification. Furthermore, the data set was changed between this contribution and both Schlosser et al (2019) and Beuth et al (2020), to make our approach more applicable to the process.…”

“…Our other previous contribution of Beuth et al (2020) represents an approach with the different data set and based on a biologically plausible model of visual attention. The previous work's system (Beuth et al 2020) is a neuro-computational model deeply rooted within the neuroscience of the brain, including neuronal firing rates and expressed human behavior (Beuth 2019). The approach of Beuth et al (2020) has allowed us to analyze the idea of attention deeply, which was one of the objectives of our previous contribution.…”

“…This sampling process is followed by a data augmentation to generate additional faulty samples. Despite several images being identical in the data set, the images appear differently while producing a good amount of image variations based on one source image (Beuth et al 2020;Zhao and Kumar 2018).…”

“…On each side, the center of the chip border is determined by the pipeline and returned as the center of the street region. We define a region of interest (ROI) of 120% the size of the chip and 6× the width of the street around this center, and follow the in Beuth et al (2020) outlined procedure. The goal of our application is to detect dicing defects which occur in the space between the streets and the chips and possibly continue inside the chips.…”

A Systematic Review of Deep Learning for Silicon Wafer Defect Recognition

“…With the rapid development of technology in recent years, deep learning as a computer-aided algorithm has been actively realized in many fields [14]. Although it has been serving the semiconductor industry in many areas [15]- [17], deep learning's contribution in detecting and analyzing fabrication flaws is tremendous. Fabrication process defects appear in the form of specific patterns on the silicon substrate.…”