Geometrical interpretation and improvements of the Blahut-Arimoto's algorithm

Naja, Z.; Alberge, Florence; Duhamel, Pierre

doi:10.1109/icassp.2009.4960131

Cited by 10 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of iterations required by our algorithm to obtain an a priori error of ε is O ε −1 log N , where N denotes the input dimension. Moreover, one can construct an accelerated version of the algorithm similar to the classical case [24], [25] to lower the number of iterations required by a constant factor compared to the standard version. This procedure also gives rise to heuristics that provide a significant speed-up of the algorithm in our numerics.…”

Section: Overview Of Resultsmentioning

confidence: 99%

“…Blahut-Arimoto type algorithms are specifically tailored for entropic problems and have analytic convergence guarantees derived from entropy inequalities (see [16], [17], [18], [19], [20], [21], [22] for classical extensions of the original works). Accelerated Blahut-Arimoto algorithms with faster analytical and numerical convergence are also known [23], [24], [25], [26]. While there have been works on Blahut-Arimoto inspired algorithms to calculate the capacity of classical-quantum channels [3], [27], this work generalizes the Blahut-Arimoto algorithm to the fully quantum setting.…”

Section: Introductionmentioning

confidence: 99%

“…Fig.1. Convergence of the Blahut-Arimoto algorithm to the coherent information of the amplitude damping channel E AD 0.3 given in(25) in the standard and adaptive accelerated case with acceleration parameter γ (t) in the t-th iteration step as defined in(11). The figure shows the lower bound on the coherent information in each iteration step t until the a posteriori bound given in Proposition 3.6 ensures that we terminate when |C − C(t)| ≤ 10 −6 .…”

mentioning

confidence: 99%

See 2 more Smart Citations

Quantum Blahut-Arimoto Algorithms

Ramakrishnan

Iten

Scholz

et al. 2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

We generalize alternating optimization algorithms of Blahut-Arimoto type to the quantum setting. In particular, we give iterative algorithms to compute the mutual information of quantum channels, the thermodynamic capacity of quantum channels, the coherent information of less noisy quantum channels, and the Holevo quantity of classical-quantum channels. Our convergence analysis is based on quantum entropy inequalities and leads to a priori additive ε-approximations after O ε −1 log N iterations, where N denotes the input dimension of the channel. We complement our analysis with an a posteriori stopping criterion which allows us to terminate the algorithm after significantly fewer iterations compared to the a priori criterion in numerical examples. Finally, we discuss heuristics to accelerate the convergence.

show abstract

Section: Overview Of Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Quantum Blahut-Arimoto Algorithms

Ramakrishnan

Iten

Scholz

et al. 2020

2020 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…Finally, we note that the Blahut-Arimoto algorithm [Blahut, 1972, Arimoto, 1972 proves to be a critical algorithmic tool for enjoying the practical virtues of rate-distortion theory in sequential decision-making problems. The algorithm itself has been a popular object of study for its utility in the information-theory community as well as its efficacy as an alternating optimization algorithm [Boukris, 1973, Rose, 1994, Sayir, 2000, Matz and Duhamel, 2004, Niesen et al, 2007, Vontobel et al, 2008, Naja et al, 2009, Yu, 2010]. While we do not explore any extensions of the Blahut-Arimoto algorithm in this work due to their focus on improving computational efficiency, practitioners may find these computational advantages meaningful and potentially necessary when implementing and deploying BLAIDS for real-world applications.…”

Section: A Related Workmentioning

confidence: 99%

The Value of Information When Deciding What to Learn

Arumugam¹,

Roy²

2021

Preprint

View full text Add to dashboard Cite

All sequential decision-making agents explore so as to acquire knowledge about a particular target. It is often the responsibility of the agent designer to construct this target which, in rich and complex environments, constitutes a onerous burden; without full knowledge of the environment itself, a designer may forge a suboptimal learning target that poorly balances the amount of information an agent must acquire to identify the target against the target's associated performance shortfall. While recent work has developed a connection between learning targets and rate-distortion theory to address this challenge and empower agents that decide what to learn in an automated fashion, the proposed algorithm does not optimally tackle the equally important challenge of efficient information acquisition. In this work, building upon the seminal design principle of information-directed sampling [Russo and Van Roy, 2014], we address this shortcoming directly to couple optimal information acquisition with the optimal design of learning targets. Along the way, we offer new insights into learning targets from the literature on rate-distortion theory before turning to empirical results that confirm the value of information when deciding what to learn.Recent work has developed a connection between rate-distortion theory [Shannon, 1959, Berger, 1971 and the problem of deciding what an agent should learn [Arumugam and Van Roy, 2021], optimally balancing between the requisite information needed by any agent to identify a learning target and the corresponding performance shortfall associated with this target. Intuitively, the complexity of a learning problem can be quantified by the bits of information an agent must acquire from the 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

“…The capacity achieving input distribution λ * is the fixed point of the mapping F (λ) in (12). That is, λ * = F (λ * ).…”

Section: Lemmamentioning

confidence: 99%

Analysis for the Slow Convergence in Arimoto Algorithm

Nakagawa

Takei

Watabe

2018

2018 International Symposium on Information Theory and Its Applications (ISITA)

View full text Add to dashboard Cite

In this paper, we investigate the convergence speed of the Arimoto algorithm. By analyzing the Taylor expansion of the defining function of the Arimoto algorithm, we will clarify the conditions for the exponential or 1/N order convergence and calculate the convergence speed. We show that the convergence speed of the 1/N order is evaluated by the derivatives of the Kullback-Leibler divergence with respect to the input probabilities. The analysis for the convergence of the 1/N order is new in this paper. Based on the analysis, we will compare the convergence speed of the Arimoto algorithm with the theoretical values obtained in our theorems for several channel matrices.

show abstract

Geometrical interpretation and improvements of the Blahut-Arimoto's algorithm

Cited by 10 publications

References 9 publications

Quantum Blahut-Arimoto Algorithms

Quantum Blahut-Arimoto Algorithms

The Value of Information When Deciding What to Learn

Analysis for the Slow Convergence in Arimoto Algorithm

Contact Info

Product

Resources

About