TSSTNet: A Two-Stream Swin Transformer Network for Salient Object Detection of No-Service Rail Surface Defects

Abstract

The detection of no-service rail surface defects is important in the rail manufacturing process. Detection of defects can prevent significant financial losses. However, the texture and form of the defects are often very similar to the background, which makes them difficult for the human eye to distinguish. How to accurately identify rail surface defects thus poses a challenge. We introduce salient object detection through machine vision to deal with this challenge. Salient object detection locates the most “significant” areas of an image using algorithms, which constitute an integral part of machine vision inspection. However, existing saliency detection networks suffer from inaccurate positioning, poor contouring, and incomplete detection. Therefore, we propose an innovative deep learning network named Two-Stream Swin Transformer Network (TSSTNet) for salient detection of no-service rail surface defects. Specifically, we propose a two-stream encoder—one stream for feature extraction and the other for edge extraction. TSSTNet also includes a three-stream decoder, consisting of a saliency stream, edge stream, and fusion stream. For the problem of incomplete detection, we innovatively introduce the Swin Transformer to model global information. For the problem of unclear contours, we expect to deepen the understanding of the difference in depth between the foreground and background through the learning of contour maps, so the contour alignment module (CAM) is created to deal with this problem. Moreover, to make the most of multimodal information, we suggest a multi-feature fusion module (MFFM). Finally, we conducted comparative experiments with 10 state-of-the-art (SOTA) approaches on the NRSD-MN datasets, and our model performed more competitively than others on five metrics.

Deep Learning for Automated Visual Inspection in Manufacturing and Maintenance: A Survey of Open- Access Papers

“…Most of these publications yielded results within the top range of their AVI use case, as has been shown in Section 4 [70,123,152,164,187]; therefore, we expect and recommend the acceleration of the application of vision transformers to the domain of AVI, especially even newer, very parameter-efficient models like Dino and its variants [286][287][288][289]. All publications utilizing transformer models applied them via supervised transfer learning, disregarding their excellent semi-supervised learning capabilities with regard to generalization and data efficiency.…”

“…Three authors decided to approach it with DL-based models of the Transformer family. For example, Wan et al [123] utilized a Swin-Transformer to localize damages on rail surfaces. Completeness checks can be executed with YOLO and/or regional convolutional neural networks (RCNNs).…”

Shrimp Oil-Enriched Mayonnaise Prepared Using Fish Myofibrillar Protein as a Substitute for Egg Yolk: Physical, Rheological, and Sensory Properties

“…Patil and Benjakul [13] reported that increasing the levels of fish oil in a virgin coconut oil mayonnaise increased the a* and b* values and reduced the L*, owing to the presence of some indigenous pigments in fish oil. Generally, a difference in color was visually perceived when ∆E* was above 3.7 [40]. Among all the samples, SO10:FMP0 had higher ∆E* and ∆C*, while SO5:FMP50 showed the lowest values when compared to the SBO sample (p < 0.05).…”

Steel Surface Defect Recognition: A Survey

“…Second, there are many types of steel surface defects and some of these defects may overlap, while most classification tasks can only find the defects with the highest confidence level in the defect category, resulting in imprecise classification results [3]. In the actual production environment, it is very difficult to obtain high quality datasets for training machine learning related algorithms because defects in steel production are originally small probability events, and it is very difficult to obtain many samples with various types of defects; the labeling of the data is also costly and labor intensive [4][5][6][7]. Since the classification of defect categories is based on human subjective classification and there is still no strict classification standard, these lead to difficulties in the progress of classification work.…”

“…The specific division structure of the steel surface types is shown in Figure 1. lated algorithms because defects in steel production are originally small pro events, and it is very difficult to obtain many samples with various types of def labeling of the data is also costly and labor intensive [4][5][6][7]. Since the classification o categories is based on human subjective classification and there is still no strict cl tion standard, these lead to difficulties in the progress of classification work.…”