Extension of the Blahut–Arimoto Algorithm for Maximizing Directed Information

Naiss, Iddo; Permuter, Haim H.

doi:10.1109/tit.2012.2214202

Cited by 37 publications

(44 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This form is the negative of a concave function, as proven in [12,Lemma 2]. Furthermore, in the same lemma, we show that the directed information is continuous with continuous partial derivatives; similar proof applies here.…”

Section: Derivation Of Algorithmsupporting

confidence: 74%

“…Clearly, the probability of each subset, , is since the left-hand side is a summation of the probabilities of all trees with the same branch associated with , and we are left with the probability of that one branch. Now, for every and due to the definition of , we have Therefore (12) and we obtain that where (a) follows the inequality in (12). We now use the inequality , which is true for all [2, eq.…”

Section: Achievability Proof (Theorem 1)mentioning

confidence: 99%

“…The alternating maximization procedure is described in [12] by two maximization functions operating on the two-variable objective , where . The first is , which is the one that achieves , and the second is , which is the point that achieves .…”

Section: Derivation Of Algorithmmentioning

confidence: 99%

“…In [12,Lemma 6], we provided another estimator for the limit of the th directed information. This was the directed information rate, which served as an estimator for the feedback channel capacities.…”

Section: B Markov Source and Single-letter Distortionmentioning

confidence: 99%

See 3 more Smart Citations

Computable Bounds for Rate Distortion With Feed Forward for Stationary and Ergodic Sources

Naiss

Permuter

2013

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

In this paper, we consider the rate distortion problem of discrete-time, ergodic, and stationary sources with feed forward at the receiver. We derive a sequence of achievable and computable rates that converge to the feed-forward rate distortion. We show that for ergodic and stationary sources, the rate is achievable for any , where the minimization is performed over the transition conditioning probability such that . We also show that the limit of exists and is the feed-forward rate distortion. We follow Gallager's proof where there is no feed forward and, with appropriate modification, obtain our result. We provide an algorithm for calculating using the alternating minimization procedure and present several numerical examples. We also present a dual form for the optimization of and transform it into a geometric programming problem.Index Terms-Alternating minimization procedure, Blahut-Arimoto (BA) algorithm, causal conditioning, concatenating code trees, directed information, ergodic and stationary sources, ergodic modes, geometric programming (GP), rate distortion with feed forward.

show abstract

Section: Derivation Of Algorithmsupporting

confidence: 74%

Section: Achievability Proof (Theorem 1)mentioning

confidence: 99%

Section: Derivation Of Algorithmmentioning

confidence: 99%

Section: B Markov Source and Single-letter Distortionmentioning

confidence: 99%

See 2 more Smart Citations

Computable Bounds for Rate Distortion With Feed Forward for Stationary and Ergodic Sources

Naiss

Permuter

2013

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…Computation of (4) is possible by, e.g., the algorithm in [9], which is a combination of Blahut-Arimoto algorithm and dynamic programming.…”

Section: Theoremmentioning

confidence: 99%

Capacity of the discrete memoryless energy harvesting channel with side information

Özel

Tutuncuoglu

Ulukuş

et al. 2014

2014 IEEE International Symposium on Information Theory

View full text Add to dashboard Cite

Abstract-We determine the capacity of a discrete memoryless communication channel with an energy harvesting transmitter and its battery state information available at the transmitter and the receiver. This capacity is an upper bound for the problem where side information is available only at the transmitter. Since channel output feedback does not increase the capacity in this case, we equivalently study the resulting finite-state Markov channel with feedback. We express the capacity in terms of directed information. Additionally, we provide sufficient conditions under which the capacity expression is further simplified to include the stationary distribution of the battery state. We also obtain a single-letter expression for the capacity with battery state information at both sides and an infinite-sized battery. Lastly, we consider achievable schemes when side information is available only at the transmitter for the case of an arbitrary finite-sized battery. We numerically evaluate the capacity and achievable rates with and without receiver side information.

show abstract

Information rate analysis of ASK-based molecular communication systems with feedback

Ghavami

Adve

Lahouti

2021

Nano Communication Networks

View full text Add to dashboard Cite

Extension of the Blahut–Arimoto Algorithm for Maximizing Directed Information

Cited by 37 publications

References 29 publications

Computable Bounds for Rate Distortion With Feed Forward for Stationary and Ergodic Sources

Computable Bounds for Rate Distortion With Feed Forward for Stationary and Ergodic Sources

Capacity of the discrete memoryless energy harvesting channel with side information

Information rate analysis of ASK-based molecular communication systems with feedback

Contact Info

Product

Resources

About