Multi-Modal Data Fusion Using Deep Neural Network for Condition Monitoring of High Voltage Insulator

Abstract

A novel Fusion Convolutional Network (FCN) is proposed in this research for potential realtime monitoring of insulators using unmanned aerial vehicle (UAV) edge devices. Precise airborne imaging of outdoor objects, such as high voltage insulators, suffers from varied object resolution, cluttered backgrounds, unclear or contaminated surfaces, and illumination conditions. Accurate information about the insulator surface condition is essential and is of a high priority since insulator breakdown is a leading cause of electrical failure. A multi-modal information fusion (MMIF) system is developed during this research to analyze and classify possible contaminations present on the electrical insulators. A novel system, referred to as FCN, consists of a Convolutional Neural Network (CNN) and a binary Multilayer Neural Network (MNN) sub-classifier. While constructing the MMIF dataset for training and testing the novel FCN, the image classification output of the CNN is combined with the leakage current values (LCV) obtained as the classification output of MNN. Each sample of the MMIF dataset is, therefore, represented as a series of fusions. Later, sub-classifiers, of the FCN, are trained to identify the contamination types in the fusion series by implementing a voting system of sub-classifiers which is trained to identify a given class. As a result of the implementation of the proposed FCN, the classification accuracy increased by 8.4%, i.e., from 92% to 99.76%. To compare and benchmark the performance of proposed FCN, conventional classification algorithms are also implemented on the fusion of features that are extracted employing the wavelet transform and PCA methods. State-of-the-art CNN architectures are also discussed on account of their time consumption and memory usage. The conceptualization of a potential hardware implementation of the proposed FCN, on emerging edge devices, is also provided for completeness of the discussion. Pertinent outcomes of this research can be further extended to other potential applications of airborne imaging. INDEX TERMS Convolutional Neural Network, multi-modal information fusion, electrical insulators, unmanned aerial vehicle.

Deep Learning in High Voltage Engineering: A Literature Review

“…In general, the literature shows an inadequate amount of work that has been devoted to train deep network architectures to classify and predict pollution levels non-intrusively. All the work done is either focused on applying deep learning models to intrusive measurement techniques [130,131] or applying classical machine learning using non-intrusive techniques [132][133][134][135]. More research is needed to combine deep learning models with non-intrusive approaches for monitoring, particularly those based on radiation type sensors.…”

Few-shot contrastive learning for image classification and its application to insulator identification

“…The triplet loss pushes the negative sample outside of the boundary by the margin and keeps the positive sample within the boundary. (6) The N-pair loss [57] is a generalization of triplet loss. It identifies a positive sample by comparing more than one negative sample, as shown in (6).…”

“…(6) The N-pair loss [57] is a generalization of triplet loss. It identifies a positive sample by comparing more than one negative sample, as shown in (6). z i , z j , z k are the anchor vector, negative vectors, and positive vector, respectively.…”

“…In the wake of superior performance in feature extraction and representation, deep learning has also been introduced into image recognition [4,5]. Mussina et al [6] consider integrating multimodal information fusion for classifying possible situations by a fully connected network. Jiang et al [7] propose multi-level perception to capture global and local representative features.…”

Classification of insulators using neural network based on computer vision

“…The use of advanced models to identify defects in insulators has been studied by several researchers. The convolutional neural network (CNN) architecture can present results of accuracy of up to 99.76% for the monitoring of insulators from aerial images [16]. For the detection of defects in insulators, modern combined models such as the ResNeSt and region pro-posal network (RPN) can be used.…”