FogSurv: A Fog-Assisted Architecture for Urban Surveillance Using Artificial Intelligence and Data Fusion

Abstract

Urban surveillance, of which airborne urban surveillance is a vital constituent, provides situational awareness (SA) and timely response to emergencies. The significance and scope of urban surveillance has increased manyfold in recent years due to the proliferation of unmanned aerial vehicles (UAVs), Internet of things (IoTs), and multitude of sensors. In this article, we propose FogSurv-a fogassisted surveillance architecture and framework leveraging artificial intelligence (AI) and information/data fusion for enabling real-time SA and monitoring. We also propose an AI-and data-driven information fusion model for FogSurv to help provide (near) real-time SA, threat assessment, and automated decisionmaking. We further present a latency model for AI and information fusion processing in FogSurv. We then discuss several use cases of FogSurv that can have a huge impact on multifarious fronts of national significance ranging from safeguarding national security to monitoring of critical infrastructures. We conduct an extensive set of experiments to demonstrate that FogSurv using AI and data fusion help provide near real-time inferences and SA. Experimental results demonstrate that FogSurv provides a latency improvement of 37% on average over cloud architectures for the selected benchmarks. Results further indicate that combining AI with data fusion as in FogSurv can provide a speedup of up to 9.8× over AI without data fusion while also maintaining or improving the inference accuracy. Additionally, results show that AI combined with fusion of different image modalities obtained through UAVs in FogSurv results in improved average precision of target detection for surveillance as compared to AI without data fusion for different target scales and environment complexity.

Human-machine cooperative AI decision-making with heterogeneous data

“…Cyber physical systems (CPS) include many sensors connected on a cyber network. A CPS architecture could consist of cloud [44,45], fog [46], and edge [47,48] services. These different network considerations are important as they could have different DL principle requirements such as security, safety and privacy On the network, different results are tradeoffs such as the distance changing the timeliness, the proximity determining the quality of the data for accuracy, and the memory implications for the amount of data to determine the credibility such as in blockchain methods.…”

Distribution of DDS-cerberus authenticated facial recognition streams

“…The first paper to inspire the experiment design was Munir et al's research on proposing FogSurv, a fog-assisted architecture to be used in urban areas for real-time surveillance using artificial intelligence (AI) [13]. They constructed a centralized cloud server with fog nodes to offload communication and computation power burdens.…”

“…Remotely operated bases use forward-deployed UAVs to support battlespace surveillance in contested environments [1]. The inter-communication links between UAVs and external links to cloud support services need to be robust enough to send and process video and images given terrain diversity and secure enough to thwart adversary attacks [2,3]. Other messages could have QoS as best effort when a system can handle not receiving every message.…”

Blockchain-Aware Distributed Dynamic Monitoring: A Smart Contract for Fog-Based Drone Management in Land Surface Changes

“…These data are sent from drone-based wireless captured sensors to fog nodes where they are measured for performance accuracy [35]. It can support more edge nodes, allowing for more efficient drone deployment and node accessibility [36,37]. This includes data relay, data-to-data communication, low latency, high data transmission rate, and other technical support for other devices connected to the fog node [38].…”