Capacity of Finite State Channels with Feedback: Algorithmic and Optimization Theoretic Properties

Grigorescu, Andrea; Boche, Holger; Schaefer, Rafael F.; Poor, H. Vincent

doi:10.48550/arxiv.2201.11639

Cited by 3 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We stack the LSTM network with additional fully-connected (FC) networks to increase the expressiveness of the model. The output layer of h φ is given by an m-dimensional softmax activation (17). Denoting the LSTM and FC maps by g φ 1 and g φ 2 , respectively, the PMF generator output evolution is given by…”

Section: Pmf Generatormentioning

confidence: 99%

See 1 more Smart Citation

Optimizing Estimated Directed Information over Discrete Alphabets

Tsur

Aharoni

Goldfeld

et al. 2022

2022 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

Directed information (DI) is a fundamental measure for the study and analysis of sequential stochastic models. In particular, when optimized over input distributions it characterizes the capacity of general communication channels. However, analytic computation of DI is typically intractable and existing optimization techniques over discrete input alphabets require knowledge of the channel model, which renders them inapplicable when only samples are available. To overcome these limitations, we propose a novel estimation-optimization framework for DI over discrete input spaces. We formulate DI optimization as a Markov decision process and leverage reinforcement learning techniques to optimize a deep generative model of the input process probability mass function (PMF). Combining this optimizer with the recently developed DI neural estimator, we obtain an end-to-end estimation-optimization algorithm which is applied to estimating the (feedforward and feedback) capacity of various discrete channels with memory. Furthermore, we demonstrate how to use the optimized PMF model to (i) obtain theoretical bounds on the feedback capacity of unifilar finite-state channels; and (ii) perform probabilistic shaping of constellations in the peak power-constrained additive white Gaussian noise channel.

show abstract

Section: Pmf Generatormentioning

confidence: 99%

“…A finite-state channel is unifilar if its state evolves as a time-invariant deterministic function of the past input, output, and state tuple.2 or when the feedback capacity itself is not algorithmically (Borel-Turing) computable[17].…”

mentioning

confidence: 99%

Optimizing Estimated Directed Information over Discrete Alphabets

Tsur

Aharoni

Goldfeld

et al. 2022

2022 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…However, this optimization is challenging since analytic computation of DI requires knowledge of the underlying probability law, which is typically unavailable in practice. Furthermore, even when the probability law is given, tractable DI characterizations that lend well for optimization are rare [17,18], as it is generally given by a multiletter expression. To address this, the goal of the paper is to develop a computable and provably accurate estimate of DI.…”

Section: Introductionmentioning

confidence: 99%

Neural Estimation and Optimization of Directed Information over Continuous Spaces

Tsur¹,

Aharoni²,

Goldfeld³

et al. 2022

Preprint

View full text Add to dashboard Cite

This work develops a new method for estimating and optimizing the directed information rate between two jointly stationary and ergodic stochastic processes. Building upon recent advances in machine learning, we propose a recurrent neural network (RNN)-based estimator which is optimized via gradient ascent over the RNN parameters. The estimator does not require prior knowledge of the underlying joint and marginal distributions. The estimator is also readily optimized over continuous input processes realized by a deep generative model. We prove consistency of the proposed estimation and optimization methods and combine them to obtain end-to-end performance guarantees. Applications for channel capacity estimation of continuous channels with memory are explored, and empirical results demonstrating the scalability and accuracy of our method are provided. When the channel is memoryless, we investigate the mapping learned by the optimized input generator.

show abstract

“…The algorithmic computability properties of capacity has been studied for various channels, including finite-state channels (FSCs) [7], FSCs with feedback [8], compound channels [9], and correlation-assisted DMCs [10]. For all of these channels, it has been demonstrated that the capacities are not generally computable functions, due to their complicated descriptions.…”

mentioning

confidence: 99%

Algorithmic Computability and Approximability of Capacity-Achieving Input Distributions

Boche

Schaefer

Poor

2023

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

Designing capacity-achieving coding schemes for the band-limited additive colored Gaussian noise (ACGN) channel has been and is still a challenge. In this paper, the capacity of the band-limited ACGN channel is studied from a fundamental algorithmic point of view by addressing the question of whether or not the capacity can be algorithmically computed. To this aim, the concept of Turing machines is used, which provides fundamental performance limits of digital computers. t is shown that there are band-limited ACGN channels having computable continuous spectral densities whose capacity are non-computable numbers. Moreover, it is demonstrated that for those channels, it is impossible to find computable sequences of asymptotically sharp upper bounds for their capacities. I. INTRODUCTIONWhile the 5th generation (5G) of mobile networks enable the industrial application of the Internet of Things (IoT), the technological advances of the wireless network infrastructure aimed for 6G will provide

show abstract

Capacity of Finite State Channels with Feedback: Algorithmic and Optimization Theoretic Properties

Cited by 3 publications

References 30 publications

Optimizing Estimated Directed Information over Discrete Alphabets

Optimizing Estimated Directed Information over Discrete Alphabets

Neural Estimation and Optimization of Directed Information over Continuous Spaces

Algorithmic Computability and Approximability of Capacity-Achieving Input Distributions

Contact Info

Product

Resources

About