A Heterogeneous Edge-Fog Environment Supporting Digital Twins for Remote Inspections

Abstract

The increase in the development of digital twins brings several advantages to inspection and maintenance, but also new challenges. Digital models capable of representing real equipment for full remote inspection demand the synchronization, integration, and fusion of several sensors and methodologies such as stereo vision, monocular Simultaneous Localization and Mapping (SLAM), laser and RGB-D camera readings, texture analysis, filters, thermal, and multi-spectral images. This multidimensional information makes it possible to have a full understanding of given equipment, enabling remote diagnosis. To solve this problem, the present work uses an edge-fog-cloud architecture running over a publisher-subscriber communication framework to optimize the computational costs and throughput. In this approach, each process is embedded in an edge node responsible for prepossessing a given amount of data that optimizes the trade-off of processing capabilities and throughput delays. All information is integrated with different levels of fog nodes and a cloud server to maximize performance. To demonstrate this proposal, a real-time 3D reconstruction problem using moving cameras is shown. In this scenario, a stereo and RDB-D cameras run over edge nodes, filtering, and prepossessing the initial data. Furthermore, the point cloud and image registration, odometry, and filtering run over fog clusters. A cloud server is responsible for texturing and processing the final results. This approach enables us to optimize the time lag between data acquisition and operator visualization, and it is easily scalable if new sensors and algorithms must be added. The experimental results will demonstrate precision by comparing the results with ground-truth data, scalability by adding further readings and performance.

An Edge-Fog Architecture for Distributed 3D Reconstruction and Remote Monitoring of a Power Plant Site in the Context of 5G

“…In remote distributed 3D reconstruction, low-level filters and high-level accumulating processes clearly separate data and information; raw high-resolution images and point clouds can be filtered in the individual embedded nodes, whereas asynchronous stacking can be carried out in more centralized, powerful hardware. The proposed approach breaks down the problem into different layers considering the amount of data, computing power, sensor proximity, and network throughput [19].…”

“…The current literature in 3D reconstruction shows that most of the research focuses on optimizing the reconstruction quality in a centralized manner [11,12], new reconstruction approaches [13][14][15], and enhancing algorithms performance [16]. Only a few studies have focused on developing a scalable distributed system regarding remote 3D supervision [17][18][19]. In many cases, the 3D processing application is located in the cloud.…”

“…Pinto et al [20] detailed a mathematical simulation in order to study and analyze an edge-fog-cloud architecture in multi-robots. The authors of Silva et al [19] presented a real case study for a 3D reconstruction approach that uses an edge-fog architecture. Different from them, this research work analyzes the cost of each node process, considering more than one real robot as an Edge Node in the architecture.…”

Sensor Information Sharing Using a Producer-Consumer Algorithm on Small Vehicles

“…There are different examples of applications that use ROS in its architecture as in [ 28 , 29 , 30 ]. In [ 31 ] Apache Kafka was used to parallelize a set learning architecture.…”

“…Some architectures propose using cloud services to improve the computational capacity and intelligence in vehicles such as AGVs [ 35 ]. Some examples suggest using architectures based on the edge-fog model as seen in [ 30 ]. The authors used the publish-subscribe model of ROS to perform 3D scenario reconstruction through SLAM and proof of concept presented in [ 36 ].…”