Analysis of Depth and Semantic Mask for Perceiving a Physical Environment Using Virtual Samples Generated by a GAN

Abstract

Micro aerial vehicles (MAVs) can make explorations in 3D environments using technologies capable of perceiving the environment to map and estimate the location of objects that could cause collisions, such as Simultaneous Localization and Mapping (SLAM). Nevertheless, the agent needs to move during the environment mapping, reducing the flying time to employ additional activities. It has to be noted that adding more devices (sensors) to MAVs implies more power consumption. Since more energy to perform tasks is required, growing the dimensions of MAVs limits the flying time. Contrarily, Generative Adversarial Networks (GAN) have demonstrated the usefulness of creating images from one domain to another, but the GAN domain changes require a large number of samples. Therefore, an interoperability coefficient is employed to determine a minimum number of samples to connect the different domains. In order to prove the coefficient, the performance to estimate the depth and semantic mask between authentic and virtual samples with the number limited of samples is analyzed. Consequently, an RGB-D sensor can be replaced by a few samples of a real scenario based on GANs. Although GAN allows creating images with depth and semantic mask information, there is an additional problem to be tackled: the presence of intrinsic noise, where a simple GAN architecture is not enough. In this proposal, the performance of this solution against a physical RGB-D sensor (Microsoft Kinect V1) and other state-of-the-art approaches is compared. Experimental results allow us to affirm that this proposal is a viable option to replace a physical RGB-D sensor with limited information.

A Comprehensive Review on Artificial Intelligence/Machine Learning Algorithms for Empowering the Future IoT Toward 6G Era

“…A pose estimation error minimization approach is studied in [40] to accurately extract the features of the tracked objects and environments. Energy effective SLAM operations may be ensured by analyzing the minimum number of samples required to be collected by the sensors/cameras for effective reconstruction of the map images, which is done in [41].…”

Path Generator with Unpaired Samples Employing Generative Adversarial Networks

“…Existing evidence where ML replaces specialized smartphone sensors, as described in [ 31 ], is used to accurately replace a depth sensor from an image taken from the device. Likewise, another alternative solution is to employ simulators and ML approaches to connect a sample-limited environment with a known environment [ 32 ]. Thus, ML techniques provide novel features to the current systems.…”

Sample Generation Method Based on Variational Modal Decomposition and Generative Adversarial Network (VMD–GAN) for Chemical Oxygen Demand (COD) Detection Using Ultraviolet Visible Spectroscopy

“…16 Recently, the generative adversarial network (GAN) has received increased attention for sample generation. 17 Maldonado-Romo et al 18 used the GAN to create images with depth and semantic mask information to model a physical environment. Chaudhari et al 19 proposed a modified generator GAN wherein the generator was fed with original data and multivariate noise to augment the gene expression data set.…”