Reidentification of Objects From Aerial Photos With Hybrid Siamese Neural Networks

Abstract

In this article, we consider the task of reidentifying the same object in different photos taken from separate positions and angles during aerial reconnaissance, which is a crucial task for the maintenance and surveillance of critical large-scale infrastructure. To effectively hybridize deep neural networks with available domain expertise for a given scenario, we propose a customized pipeline, wherein a domain-dependent object detector is trained to extract the assets (i.e., subcomponents) present on the objects, and a siamese neural network learns to reidentify the objects, exploiting both visual features (i.e., the image crops corresponding to the assets) and the graphs describing the relations among their constituting assets. We describe a realworld application concerning the reidentification of electric poles in the Italian energy grid, showing our pipeline to significantly outperform siamese networks trained from visual information alone. We also provide a series of ablation studies of our framework to underline the effect of including topological asset information in the pipeline, learnable positional embeddings in the graphs, and the effect of different types of graph neural networks on the final accuracy.

Applied Control and Artificial Intelligence for Energy Management: An Overview of Trends in EV Charging, Cyber-Physical Security and Predictive Maintenance

“…Pole mapping on overhead power lines Mapping out all the poles of the networks with cyclically planned aerial flights [51] Pole mapping on the generic power grid Mapping out all the poles of the networks with cyclically planned aerial flights [52] Power line corridor surveys Algorithm for automatic reidentification of the same object from different pictures [53] Mission of aerial reconnaissance for the reidentification of electric poles in the Italian power grid Deep learning-based strategy for reidentifying the same object in different photos taken from separate positions and angles [56] Predictive maintenance for photovoltaic power plants Data-driven toolkit [62] Predictive maintenance for photovoltaic power plants Data-driven approach based on sensor network analysis for unveiling hidden precursors in failure modes [63] Anomaly detection for the 3SUN solar cell production plant in Catania, Italy Robust anomaly detection using Monte Carlo-based pre-processing [64] Fault detection for wind turbines from SCADA data…”

“…In a recent study by Devoto et al [56], a deep learning-based strategy for reidentifying objects in aerial reconnaissance missions is proposed. This strategy is crucial for maintaining and monitoring critical infrastructures.…”