Euclidean Information Theory of Networks

Huang, Shao-Lun; Suh, Changho; Zheng, Lizhong

doi:10.1109/tit.2015.2484066

Cited by 20 publications

(11 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our main results relate mutual information and Holevo quantity with Fisher information and its quantum counterparts. In the classical case we show that in the weak estimation regime [48,49]…”

Section: Weak Estimation Regimementioning

confidence: 99%

Super-additivity in communication of classical information through quantum channels from a quantum parameter estimation perspective

2017

View full text Add to dashboard Cite

We point out a contrasting role the entanglement plays in communication and estimation scenarios. In the first case it brings noticeable benefits at the measurement stage (output super-additivity), whereas in the latter it is the entanglement of the input probes that enables significant performance enhancement (input super-additivity). We identify a weak estimation regime where a strong connection between concepts crucial to the two fields is demonstrated; the accessible information and the Holevo quantity on one side and the quantum Fisher information related quantities on the other. This allows us to shed new light on the problem of super-additivity in communication using the concepts of quantum estimation theory. of merit used in estimation and communication approaches are usually different [14]. The central problem of quantum communication and estimation theories is to identify the potential benefits coming from exploiting entanglement in the input states as well as at the measurement stage of the protocols. Interestingly, unlike in the communication problem, there is a great number of examples demonstrating significant gains coming from the use of entangled input probes in quantum estimation protocols, with applications ranging from optical and atomic interferometry, via magnetometry to spectroscopy and atomic clocks stabilization [17][18][19][20][21][22][23]. At the same time, in a typical estimation protocol utilizing unentangled input probes collective measurements are typically irrelevant. Thus a contrasting picture emerges: when thinking of capacities of quantum channels gains in information processing arising from utilizing entanglement are at the measurement stage whereas it is the input stage where entanglement makes a difference in the estimation scenarios.The goal of this paper is to better understand the connections between the two fields from the point of view of the super-additivity issue, understood here as a general question of utility of entanglement in estimation/ communication protocols. We show that in the weak estimation regime, where the amount of information extractable on the parameter is very small compared to the prior information, i.e. the error of estimation is large compared with the variance of prior distribution, the accessible information as well as the Holevo quantity can be expressed using Fisher information (FI)-like concepts, which allows us to discuss utility of entanglement in communication using the properties of quantities well understood within the field of quantum estimation. We also point out that in a communication problem encoding a large number of independent parameters is favored even at the cost of their limited estimation precision, which makes the estimation strategy of learning a given parameter with highest possible accuracy not likely to be useful for the communication purposes. In order to make the paper self-contained we also recall some of the recent results on applications of rate-distortion theory in quantum estimation, which is another way of connecti...

show abstract

Section: Weak Estimation Regimementioning

confidence: 99%

Super-additivity in communication of classical information through quantum channels from a quantum parameter estimation perspective

2017

View full text Add to dashboard Cite

show abstract

“…Concerning the related works, the presented paper is an extension of the main ideas briefly provided by the authors in [13], where a first approach to estimating the SMI was presented. The idea of local approximations of information measures exposed in this paper is very similar to the linear information coupling approach proposed in [5], [14], which was used there as a tool for developing insights on otherwise intractable problems in the field of communications. The study of the modal decomposition of distributions and the measurement of information through CCA in [15] is parallel to the development provided in this article.…”

Section: A Related Work and Overall Organizationmentioning

confidence: 99%

“…In this subsection, we review the property of the SMI of being a local approximation of the MI. This approximation becomes relevant for small values of dependence between random variables, which is a prominent case for applications in the context of Euclidean information theory [14].…”

Section: B Local Approximation Of Mutual Informationmentioning

confidence: 99%

“…For instance, in [35] the approximation is analyzed in the problem of coclustering contingency tables. On a more general note, we are particularly interested in the local approximations of the KL divergence and MI in [14] under the context of Linear Information Coupling (LIC) problems and Euclidean information theory. The main motivation is to translate information theory problems into linear algebra problems, thus avoiding computational and mathematical bottlenecks.…”

Section: B Local Approximation Of Mutual Informationmentioning

confidence: 99%

“…where (10) will be referred to as coherence matrix due to its intimate link with the well-known CCA tool in statistical signal processing [39]. The form of matrix C is also encountered in the areas of information theory under the context of LIC problems [14] and Hirschfeld-Gebelein-Rényi (HGR) maximal correlation concept [32], [40]. To see these links, let us express…”

Section: Discrete Sources: Second-order Statistics On the Simplex Fea...mentioning

confidence: 99%

See 2 more Smart Citations

Regularized Estimation of Information via Canonical Correlation Analysis on a Finite-Dimensional Feature Space

Cabrera

Riba

2023

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

This paper aims to estimate the information between two random phenomena by using consolidated second-order statistics tools. The squared-loss mutual information, a surrogate of the Shannon mutual information, is chosen due to its property of being expressed as a second-order moment. We first review the rationale for i.i.d. discrete sources, which involves mapping the data onto the simplex space, and we highlight the links with other well-known related concepts in the literature based on local approximations of information-theoretic measures. Then, the problem is translated to analog sources by mapping the data onto the characteristic space, focusing on the adaptability between the discrete and the analog case and its limitations. The proposed approach gains interpretability and scalability for its use on large data sets, providing a unified rationale for the free regularization parameters. Moreover, the structure of the proposed mapping allows resorting to Szegö's theorem to reduce the complexity for high dimensional mappings, exhibiting a strong duality with spectral analysis. The performance of the developed estimators is analyzed using Gaussian mixtures.

show abstract

TCGM: An Information-Theoretic Framework for Semi-supervised Multi-modality Learning

Sun

Cao

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Fusing data from multiple modalities provides more information to train machine learning systems. However, it is prohibitively expensive and timeconsuming to label each modality with a large amount of data, which leads to a crucial problem of semi-supervised multi-modal learning. Existing methods suffer from either ineffective fusion across modalities or lack of theoretical guarantees under proper assumptions. In this paper, we propose a novel information-theoretic approach -namely, Total Correlation Gain Maximization (TCGM) -for semisupervised multi-modal learning, which is endowed with promising properties: (i) it can utilize effectively the information across different modalities of unlabeled data points to facilitate training classifiers of each modality (ii) it has theoretical guarantee to identify Bayesian classifiers, i.e., the ground truth posteriors of all modalities. Specifically, by maximizing TC-induced loss (namely TC gain) over classifiers of all modalities, these classifiers can cooperatively discover the equivalent class of ground-truth classifiers; and identify the unique ones by leveraging limited percentage of labeled data. We apply our method to various tasks and achieve state-of-the-art results, including the news classification (Newsgroup dataset), emotion recognition (IEMOCAP and MOSI datasets), and disease prediction (Alzheimer's Disease Neuroimaging Initiative dataset).

show abstract

Euclidean Information Theory of Networks

Cited by 20 publications

References 20 publications

Super-additivity in communication of classical information through quantum channels from a quantum parameter estimation perspective

Super-additivity in communication of classical information through quantum channels from a quantum parameter estimation perspective

Regularized Estimation of Information via Canonical Correlation Analysis on a Finite-Dimensional Feature Space

TCGM: An Information-Theoretic Framework for Semi-supervised Multi-modality Learning

Contact Info

Product

Resources

About