Deep Joint Source-Channel Coding for Wireless Image Retrieval

Jankowski, Mikołaj; Gündüz, Deniz; Mikolajczyk, Krystian

doi:10.1109/icassp40776.2020.9054078

Cited by 44 publications

(17 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Along this line, Yang et al [25] further compressed the transmitted data by introducing the semantic relationship between feature maps and semantic concept. Jankowski et al [26] proposed a task-based compression scheme for input images for the person re-ID task, and pointed that in classification it is unnecessary to send feature maps to the receiver because the transmitter can execute the task locally and send only the predictive class label, in contrast, the transmitter can't perform image retrieval alone due to missing gallery images.…”

Section: B Framework Of Semantic Communication Systemsmentioning

confidence: 99%

“…• Based on the traditional rate-distortion theory and information bottleneck principle, an extended rate-distortion theory is proposed, which aims to optimized the communication system for the purpose of trade-off between concise representation and semantic distortion, where the semantic distortion is referred to the proper trade-off [24]- [26], (c) recently proposed multitask type semantic communication with reconstruction [4], [27]- [31], and (d) our proposed semantic communication system with reconstruction. Note that the channel codding and decoding are ignored, which is not we focus on.…”

Section: Introduction Semantic Communication Is Categorized By Shanno...mentioning

confidence: 99%

See 1 more Smart Citation

Task-Oriented Semantic Communication with Semantic Reconstruction: An Extended Rate-Distortion Theory Based Scheme

Liu¹,

Tong²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Considering the performance of intelligent task during signal exchange can help the communication system to automatically select those semantic parts which are helpful to perform the target task for compression and reconstruction, which can both greatly reduce the redundancy in signal and ensure the performance of the task. The traditional communication system based on rate-distortion theory treats all the information in the signal equally, but ignores their different importance to accomplish the task, which leads to waste of communication resources. In this paper, combined with the information bottleneck method, we present an extended rate-distortion theory which considers both concise representation and semantic distortion. Based on this theory, a task-oriented semantic image communication system is proposed. In order to verify that the proposed system can achieve performance improvement on different intelligent tasks, we apply the basic system trained with classification task to the system with object detection as the target task. The experimental results demonstrate that the proposed method outperforms the traditional and multi-task based communication system in terms of task performance at the same signal compression degree and noise interference degree. Furthermore, it is necessary to consider a compromise between rate-distortion theory and information bottleneck theory by comparing the pure rate-distortion scheme and the pure IB scheme.

show abstract

Section: B Framework Of Semantic Communication Systemsmentioning

confidence: 99%

Section: Introduction Semantic Communication Is Categorized By Shanno...mentioning

confidence: 99%

Task-Oriented Semantic Communication with Semantic Reconstruction: An Extended Rate-Distortion Theory Based Scheme

Liu¹,

Tong²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A practical edge inference problem is studied in [4], where the image of a person captured by a remote camera is to be identified within a database available at an edge server, called the reidentification (re-ID) problem. Here, the camera cannot make a local decision as it does not have access to the database.…”

Section: Distributed Inferencementioning

confidence: 99%

“…Here, the camera cannot make a local decision as it does not have access to the database. In [4], two approaches are proposed, both employing DNNs for remote inference: a task-oriented DNN-based compression scheme for digital transmission and a DNNbased analog JSCC approach,à la DeepJSCC. These schemes are compared in Fig.…”

Section: Distributed Inferencementioning

confidence: 99%

Communicate to Learn at the Edge

et al. 2020

Self Cite

View full text Add to dashboard Cite

Bringing the success of modern machine learning (ML) techniques to mobile devices can enable many new services and businesses, but also poses significant technical and research challenges. Two factors that are critical for the success of ML algorithms are massive amounts of data and processing power, both of which are plentiful, yet highly distributed at the network edge. Moreover, edge devices are connected through bandwidth-and power-limited wireless links that suffer from noise, time-variations, and interference. Information and coding theory have laid the foundations of reliable and efficient communications in the presence of channel imperfections, whose application in modern wireless networks have been a tremendous success. However, there is a clear disconnect between the current coding and communication schemes, and the ML algorithms deployed at the network edge. In this paper, we challenge the current approach that treats these problems separately, and argue for a joint communication and learning paradigm for both the training and inference stages of edge learning. I. MOTIVATIONModern machine learning (ML) techniques have made tremendous advances in areas such as machine vision, robotics, and natural language processing. Novel ML applications emerge every day, ranging from autonomous driving and finance to marketing and healthcare -potential applications are limitless. In parallel, the fifth generation (5G) of mobile technology promises to connect billions of heterogeneous devices to the network edge, supporting new applications and verticals under the banner of Internet of things (IoT). Edge devices will collect massive amounts of data, opening up new avenues for ML applications. The prevalent approach for the implementation of ML solutions on edge devices is to amass all the relevant data at a

show abstract

“…In [15] and [16], a CNN based AE is used to compress and transmit images. In [17]- [19], AE is used to compress the intermediate feature of a STL network. These deep models are trained with noise, so they are robust to channel interference.…”

Section: Introductionmentioning

confidence: 99%

Deep Joint Source-Channel Coding for Multi-Task Network

Wang,

Zhang,

et al. 2021

Preprint

View full text Add to dashboard Cite

Multi-task learning (MTL) is an efficient way to improve the performance of related tasks by sharing knowledge. However, most existing MTL networks run on a single end and are not suitable for collaborative intelligence (CI) scenarios. In this work, we propose an MTL network with a deep joint sourcechannel coding (JSCC) framework, which allows operating under CI scenarios. We first propose a feature fusion based MTL network (FFMNet) for joint object detection and semantic segmentation. Compared with other MTL networks, FFMNet gets higher performance with fewer parameters. Then FFMNet is split into two parts, which run on a mobile device and an edge server respectively. The feature generated by the mobile device is transmitted through the wireless channel to the edge server. To reduce the transmission overhead of the intermediate feature, a deep JSCC network is designed. By combining two networks together, the whole model achieves 512× compression for the intermediate feature and a performance loss within 2% on both tasks. At last, by training with noise, the FFMNet with JSCC is robust to various channel conditions and outperforms the separate source and channel coding scheme.

show abstract

Deep Joint Source-Channel Coding for Wireless Image Retrieval

Cited by 44 publications

References 10 publications

Task-Oriented Semantic Communication with Semantic Reconstruction: An Extended Rate-Distortion Theory Based Scheme

Task-Oriented Semantic Communication with Semantic Reconstruction: An Extended Rate-Distortion Theory Based Scheme

Communicate to Learn at the Edge

Deep Joint Source-Channel Coding for Multi-Task Network

Contact Info

Product

Resources

About