A Pixel-Wise Foreign Object Debris Detection Method Based on Multi-Scale Feature Inpainting

Abstract

In the aviation industry, foreign object debris (FOD) on airport runways is a serious threat to aircraft during takeoff and landing. Therefore, FOD detection is important for improving the safety of aircraft flight. In this paper, an unsupervised anomaly detection method called Multi-Scale Feature Inpainting (MSFI) is proposed to perform FOD detection in images, in which FOD is defined as an anomaly. This method adopts a pre-trained deep convolutional neural network (CNN) to generate multi-scale features for the input images. Based on the multi-scale features, a deep feature inpainting module is designed and trained to learn how to reconstruct the missing region masked by the multi-scale grid masks. During the inference stage, an anomaly map for the test image is obtained by computing the difference between the original feature and its reconstruction. Based on the anomaly map, the abnormal regions are identified and located. The performance of the proposed method is demonstrated on a newly collected FOD dataset and the public benchmark dataset MVTec AD. The results show that the proposed method is superior to other methods.

Deep Learning for Automatic Vision-Based Recognition of Industrial Surface Defects: A Survey

“…Object-oriented methods have been extensively explored and several solutions have been proposed to determine the defective region of an object. In addition to various reconstruction-based solutions [3], [211], [213], which use the one-class principle [78], [214], different alternatives can be found. For instance, Yao et al [205] developed a student-teacher network with ResNet 18 as backbone.…”

“…In such a context, the self-supervised approach learns how to reconstruct the erased regions of the input defect-free class image through feature regression. When a defective image is to be tested, it undergoes inpainting and feature prediction by the network; its abnormal regions are repaired and thus a reconstruction error map overlaying the defects is got by the difference with the original image [3].…”

“…The fusion of multi-scale features helps the network in the final decision by using both raw and semantic information to enhance localization accuracy [3]. Even in this task, fusing features to solve the tiny and low-contrast detection problem is one of the most commonly explored solutions.…”

“…(DL)-enhanced decision-making [1]. The surface of a defective product may present anomalies referrable to the composition of employed materials [2], the inclusion of foreign objects debris [3], and disturbance affecting the production phases, such as machinery and finishing processes [4] and transportation on conveyor belt [5]. Recently, massive investments have been devoted to the development of technologies that are able to automatically identify and localize defects by computing and further processing a suitable representation of raw data.…”

Feature Extraction Using Only Non-defective Images by Invariant Information Clustering and its Application to CNN Defect Detection

“…や GAN (2) を利用され たものが報告されている。Jing らの研究では，CNN から 得られた複数の特徴マップに対してマスクをかけた後， 元の画像に復元する訓練を行うことで教師なし欠陥検出 を行っている (3) 。本研究では，教師なし学習手法の一つ として注目されている不変情報クラスタリング(IIC:…”