RF-Inpainter: Multimodal Image Inpainting Based on Vision and Radio Signals

Abstract

This study demonstrates the feasibility of image inpainting using both visual information and radio frequency (RF) signals. Recent developments in imaging and vision-based technologies using RF signals have revealed the potential of leveraging multimodal information to enhance image inpainting performance. In this context, we propose RF-Inpainter-a novel inpainting method that integrates visual and wireless information by fusing defective RGB images with the received signal strength indicator (RSSI) using a deep auto-encoder model. The inpainting performance of RF-Inpainter is evaluated using experimentally obtained images and RSSI datasets in an indoor environment. Image-only inpainting and RSSI-only inpainting models are used as baselines to illustrate the superiority of RF-Inpainter over inpainting methods based on a single modality. The results establish that RF-Inpainter generates satisfactory inpainted images in most experimental scenarios, achieving a maximum improvement of 36.4% and 14.6% in terms of mean peak signal-to-noise ratio (PSNR) and mean structural similarity index (SSIM), respectively.

Point Cloud-Based Proactive Link Quality Prediction for Millimeter-Wave Communications

“…Another interesting direction for future research is to explore the inverse transformation of this study, namely, estimating spatial information from mmWave communication signals. It has already been demonstrated in the 5 GHz band that images captured by a camera installed in the same room can be estimated based on Wi-Fi channel information, such as CSI [64] or RSSI [65], in indoor environments. Furthermore, it has been shown that image generation of LOS communication paths can be achieved using 60 GHz band RSSI time series [9].…”