Semantic Communication: An Information Bottleneck View

Beck, Edgar; Bockelmann, Carsten; Dekorsy, Armin

doi:10.48550/arxiv.2204.13366

Cited by 5 publications

(9 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…distributed edge training and inference [8]) and integrated cross-layer designs will further complicate the objective function. For example, the mutual information is adopted as the objective function in semantic communications [9] to optimize the efficiencyaccuracy trade-off, which is intractable. (2) Optimization variables and model parameters of 6G optimization problems can be of high dimension due to the massive devices, large-scale antennas in wireless networks and large amounts of data in various 6G technologies and services [3].…”

Section: A Large-scale Optimization For 6gmentioning

confidence: 99%

“…The Markov chain of semantic communication thus reduces to Y ↔ X ↔ Ẑ. Since the goal of semantic communication is to maximize expected faithfulness in representing observed messages (that is to say to minimize the semantic errors) and minimize the amount of data to be transmitted [9]. Then the general form of optimization problem of semantic communication can be expressed as minimize…”

Section: Semantic Meaning Extraction and Interpretationmentioning

confidence: 99%

“…Cross-entropy losses are adopted for classification problems, such as codebook-based precoder design [195], user scheduling [73], and sub-channel selection [196], when the class labels are available. In some specific applications, information theory-based losses can help to fulfill the goals of optimization task, which can be effectively calculated using the empirical data (including training labels), such as the mutual information-based loss in semantic communication [9], min-max generative adversarial net (GAN) loss in channel estimation [197], information-bottleneck-based loss in edge inference system [198], etc. Alternatively, when training labels are unavailable, the average performance measurements are adopted to formulate the loss function for model training in an unsupervised manner [199].…”

Section: A Neural Network Design For Wireless Communicationsmentioning

confidence: 99%

“…5) End-to-End Learning for Semantic Optimization: The theory of semantic communication has caught extensive attentions in the past several years with some preliminary research results [36], [226]. In [9], the IB theory was used to design the semantic communication systems by taking the meaning of semantic information and the compression ratio into consideration. However, the IB formulation is task and label dependent, that is, the measurement quantity changes as tasks and labels change.…”

Section: B Theoretical Toolsmentioning

confidence: 99%

See 3 more Smart Citations

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Shi¹,

Lian²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

The sixth generation (6G) wireless systems are envisioned to enable the paradigm shift from "connected things" to "connected intelligence", featured by ultra high density, largescale, dynamic heterogeneity, diversified functional requirements and machine learning capabilities, which leads to a growing need for highly efficient intelligent algorithms. The classic optimization-based algorithms usually require highly precise mathematical model of data links and suffer from poor performance with high computational cost in realistic 6G applications. Based on domain knowledge (e.g., optimization models and theoretical tools), machine learning (ML) stands out as a promising and viable methodology for many complex large-scale optimization problems in 6G, due to its superior performance, generalizability, computational efficiency and robustness. In this paper, we systematically review the most representative "learning to optimize" techniques in diverse domains of 6G wireless networks by identifying the inherent feature of the underlying optimization problem and investigating the specifically designed ML frameworks from the perspective of optimization. In particular, we will cover algorithm unrolling, learning to branch-andbound, graph neural network for structured optimization, deep reinforcement learning for stochastic optimization, as well as endto-end learning for semantic optimization, for solving challenging large-scale optimization problems arising from various important wireless applications. To enable ML implementation in distributed wireless networks across massive number of end devices, federated learning for distributed optimization will further be presented. Through the in-depth discussion, we shed light on the excellent performance of ML-based optimization algorithms with respect to the classical methods, and provide insightful guidance to develop advanced ML techniques in 6G networks. Neural network design, theoretical tools of different ML methods,

show abstract

Section: A Large-scale Optimization For 6gmentioning

confidence: 99%

Section: Semantic Meaning Extraction and Interpretationmentioning

confidence: 99%

Section: A Neural Network Design For Wireless Communicationsmentioning

confidence: 99%

Section: B Theoretical Toolsmentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Shi¹,

Lian²,

Shi³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The rise of communication metrics that take the content of the message into account, such as the Value of Information (VoI) [6], represents an attempt to approach the problem in practical scenarios, and analytical studies have exploited information theory to define a semantic accuracy metric and minimize distortion [7]. In particular, information bottleneck theory [8] has been widely used to characterize Level B optimization [9]. However, translating a practical system model into a semantic space is a non-trivial issue, and the semantic problem is a subject of active research [10].…”

mentioning

confidence: 99%

Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression

Talli¹,

Pase²,

Chiariotti³

et al. 2023

Preprint

View full text Add to dashboard Cite

The coordination of robotic swarms and the remote wireless control of industrial systems are among the major use cases for 5G and beyond systems: in these cases, the massive amounts of sensory information that needs to be shared over the wireless medium can overload even high-capacity connections. Consequently, solving the effective communication problem by optimizing the transmission strategy to discard irrelevant information can provide a significant advantage, but is often a very complex task. In this work, we consider a prototypal system in which an observer must communicate its sensory data to an actor controlling a task (e.g., a mobile robot in a factory). We then model it as a remote Partially Observable Markov Decision Process (POMDP), considering the effect of adopting semantic and effective communication-oriented solutions on the overall system performance. We split the communication problem by considering an ensemble Vector Quantized Variational Autoencoder (VQ-VAE) encoding, and train a Deep Reinforcement Learning (DRL) agent to dynamically adapt the quantization level, considering both the current state of the environment and the memory of past messages. We tested the proposed approach on the well-known CartPole reference control problem, obtaining a significant performance increase over traditional approaches.

show abstract

IGZO nanofiber photoelectric neuromorphic transistors with indium ratio tuned synaptic plasticity

Zhu

Peng

Zhu

et al. 2022

Applied Physics Letters

View full text Add to dashboard Cite

Synaptic plasticity divided into long-term and short-term categories is regarded as the origin of memory and learning, which also inspires the construction of neuromorphic systems. However, it is difficult to mimic the two behaviors monolithically, which is due to the lack of time-tailoring approaches for a certain synaptic device. In this Letter, indium-gallium-zinc-oxide (IGZO) nanofiber-based photoelectric transistors are proposed for realizing tunable photoelectric synaptic plasticity by the indium composition ratio. Notably, short-term plasticity to long-term plasticity transition can be realized by increasing the ratio of indium in the IGZO channel layer. The spatiotemporal dynamic logic and low energy consumption (<100 fJ/spike) are obtained in devices with low indium ratio. Moreover, the symmetric spike-timing-dependent plasticity is achieved by exploiting customized light and electric pulse schemes. Photoelectric long-term plasticity, multi-level characteristics, and high recognition accuracy (93.5%) are emulated in devices with high indium ratio. Our results indicate that such a composition ratio modulated method could enrich the applications of IGZO nanofiber neuromorphic transistors toward the photoelectric neuromorphic systems.

show abstract

Semantic Communication: An Information Bottleneck View

Cited by 5 publications

References 23 publications

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Machine Learning for Large-Scale Optimization in 6G Wireless Networks

Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression

IGZO nanofiber photoelectric neuromorphic transistors with indium ratio tuned synaptic plasticity

Contact Info

Product

Resources

About