Distributed information-theoretic biclustering

Pichler, Georg; Piantanida, Pablo; Matz, Gerald

doi:10.1109/isit.2016.7541466

Cited by 6 publications

(7 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DSIB problem addressed in our paper is, in fact, a single letter version of the Distributed Clustering problem. The inner bound in [1] coincides with our definition of the problem. Moreover, if the Markov condition U → X → Y → Z is imposed on the multiletter variant, then those problems coincide.…”

Section: Introductionsupporting

confidence: 66%

“…Let (X, Y) be a bivariate source characterized by a fixed joint probability law P XY and consider all Markov chains U → X → Y → V. The Double-Sided Information-Bottleneck (DSIB) function is defined as [1]…”

Section: Introductionmentioning

confidence: 99%

“…An optimization algorithm was presented that intertwines both row and column clustering at all stages. Distributed Clustering from a formal information-theoretic perspective was first explicitly considered by Pichler et al [1], [6]. Consider a bivariate memoryless source with joint law P XY .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Double-Sided Information-Bottleneck Function

Dikshtein

Ordentlich

Shitz

2021

2021 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

We consider a two-terminal variant (double-sided) of the information bottleneck problem, which is related to biclustering. In our setup, X and Y are dependent random variables and the problem is to find two independent channels P U|X and P V|Y (setting the Markovian structure U → X → Y → V) that maximize I(U; V) subject to constraints on the relevant mutual information expressions: I(U; X) and I(V; Y). For jointly Gaussian X and Y, we show that Gaussian channels are optimal in the low-SNR regime, but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low, and are suboptimal for high correlation. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., X = Y), and provide supporting numerical evidence.

show abstract

Section: Introductionsupporting

confidence: 66%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Double-Sided Information-Bottleneck Function

Dikshtein

Ordentlich

Shitz

2021

2021 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

show abstract

“…This setting was studied in [12], where inner bounds and outer bounds on the achievable rate region were derived. A connection between the achievable regions of these two problems has been drawn recently in [19] via the entropy characterization. We first establish the following useful lemma, which is a generalization of [15,Lemma 3].…”

Section: Equivalence Between Ht and Identificationmentioning

confidence: 99%

Operational Equivalence of Distributed Hypothesis Testing and Identification Systems

Oechtering

Skoglund

2019

2019 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

In this paper we revisit the connections of the distributed hypothesis testing against independence (HT) problem with the Wyner-Ahlswede-Körner (WAK) problem and the identification systems (ID). We show that the strong converse for the WAK problem is equivalent to the strong converse for the HT problem via constructive and nonconstructive transformations of codes. As another consequence of the transformation we provide a new exponentially strong converse equivalence statement. Applying the same idea, we prove a new result that the-identification capacity of the ID problem is equal to the maximum-exponent of type II of error in the HT problem when both side compression is allowed.

show abstract

“…As a matter of fact, the conventional IB problem follows as a special instance of the conventional noisy lossy source coding problem [7]. Extension of this information-theoretic framework address the collaborative IB problem by Vera et al [8], and the distributed biclustering problem by Pichler et al [9]. Further connections to the problem of joint testing and lossy reconstruction has been recently studied by Katz et al [10].…”

Section: Introductionmentioning

confidence: 99%

The multi-layer information bottleneck problem

Yang

Piantanida

Gündüz

2017

2017 IEEE Information Theory Workshop (ITW)

Self Cite

View full text Add to dashboard Cite

The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variable, quantified by the mutual information. The hidden variables and the source can be arbitrarily correlated. The optimal trade-off between rates of relevance and compression (or complexity) is obtained through a singleletter characterization, referred to as the rate-relevance region. Conditions of successive refinabilty are given. Binary source with BSC hidden variables and binary source with BSC/BEC mixed hidden variables are both proved to be successively refinable. We further extend our result to Guassian models. A counterexample of successive refinability is also provided.

show abstract

Distributed information-theoretic biclustering

Cited by 6 publications

References 47 publications

The Double-Sided Information-Bottleneck Function

The Double-Sided Information-Bottleneck Function

Operational Equivalence of Distributed Hypothesis Testing and Identification Systems

The multi-layer information bottleneck problem

Contact Info

Product

Resources

About