Employing a hybrid model based on texture-biased convolutional neural networks and edge-biased vision transformers for anomaly detection of signal bonds

Abstract

The railway system of Japan plays a vital role in the national transportation network.A key issue in public transport safety is anomaly detection in railways. Lately, developing robust algorithms and methods for anomaly detection has become the premier task in this field. Recently introduced approaches based on convolutional neural networks, generative adversarial networks, and vision transformers (ViTs) have remarkably improved the research in anomaly detection. Our work proposes a high-performance module for the anomaly detection of signal bonds. First, we present an overview of the proposed module; then, the object detection model and the proposed hybrid classification model based on texture-biased convolutional neural networks and edge-biased ViTs are introduced. Finally, the proposed anomaly detection module is evaluated for accuracy using the dataset from the East Japan Railway Company and the receiver operating characteristic curve. The results show that our proposed module achieves a better performance on real-life data than the two methods that were combined to generate it, raising hope toward possible usage in other areas as well.

Deep Learning and Vision Transformer for Medical Image Analysis

“…Transformers have become the foundation for many advanced language models, such as BERT, ChatGPT [23], and T5, and have significantly advanced the capabilities of language understanding and generation systems. Vision transformers (ViTs) [24] are an adaptation of the classical transformer architecture that apply self-attention mechanisms to process image data [25], making them an exemplary powerful model for tasks in computer vision, showcasing the extension of transformers' effectiveness beyond NLP. Figure 1 shows the relationship between AI, ML, DL, and Transformers.…”