Decoding from pooled data: Phase transitions of message passing

Alaoui, A. El; Ramdas, Aaditya; Krząkała, Florent; Zdeborová, Lenka; Jordan, Michael I.

doi:10.1109/isit.2017.8007036

Cited by 9 publications

(34 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This enables us to conclude that both 1) w.h.p., provided that θ is sufficiently close to one. Finally, as we saw in Section 2.1 already, if 1) .…”

Section: Proposition 24 Whp For All I We Havementioning

confidence: 62%

“…This enables us to conclude that both 1) w.h.p., provided that θ is sufficiently close to one. Finally, as we saw in Section 2.1 already, if 1) . The second step towards Theorem 1.2 is a reduction from larger to smaller values of θ.…”

Section: Proposition 24 Whp For All I We Havementioning

confidence: 62%

“…For a start, we can safely mark as uninfected all individuals in V [i + 1] that appear in a negative test. Unfortunately, a simple calculation reveals that this will still leave us with n 1−max{(1−θ)ln 2,θ}+o (1) ≫ k undiagnosed individuals w.h.p. Thus, only a small fraction of the as yet undiagnosed individuals in V [i + 1] are actually infected.…”

Section: The Spatial Inference Vertex Cover ('Spiv') Algorithmmentioning

confidence: 99%

See 2 more Smart Citations

Optimal group testing

Coja-Oghlan,

Gebhard,

Hahn-Klimroth

et al. 2019

Preprint

View full text Add to dashboard Cite

In non-adaptive group testing we aim to identify a small set of k ∼ n θ infected individuals out of a population size n, 0 < θ < 1. We avail ourselves to a test procedure that can test a group of individuals, with the test rendering a positive result iff at least one individual in the group is infected. All tests are conducted in parallel. The aim is to devise a (possibly randomised) test design with as few tests as possible so that the infected individuals can be identified with high probability. We prove that there occurs a sharp information-theoretic/algorithmic phase transition as the number of tests passes an explicit threshold m inf . Hence, if more than (1 + ε)m inf tests are conducted, then there exist a test design and a polynomial time algorithm that identifies the set of infected individuals with high probability. By contrast, identifying the infected individuals is information-theoretically impossible with fewer than (1 − ε)m inf tests. These results resolve problems prominently posed in , Johnson et al. 2018].

show abstract

“…This enables us to conclude that both 1) w.h.p., provided that θ is sufficiently close to one. Finally, as we saw in Section 2.1 already, if 1) .…”

Section: Proposition 24 Whp For All I We Havementioning

confidence: 62%

Section: Proposition 24 Whp For All I We Havementioning

confidence: 62%

Section: The Spatial Inference Vertex Cover ('Spiv') Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Optimal group testing

Coja-Oghlan,

Gebhard,

Hahn-Klimroth

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, starting with the work of Söderberg and Shapiro [13], several works have studied the minimum number of tests required for unconstrained QNAGT as a function of k and n. In particular, Lindström [14] provided an elegant explicit, asymptotically optimal QNAGT scheme for the case k = n. More recently, the optimal number of tests for k linear in n was determined in [19], [20], and for k sublinear in n in [21], [22]. The latter work also described efficient constructions of nearly optimal QNAGT schemes (we note that [21] allows non-binary test matrices, while all other works mentioned deal with binary test matrices only).…”

Section: B Related Workmentioning

confidence: 99%

Group Testing with Runlength Constraints for Topological Molecular Storage

Agarwal¹,

Milenković²,

Pattabiraman³

et al. 2020

Preprint

View full text Add to dashboard Cite

Motivated by applications in topological DNA-based data storage, we introduce and study a novel setting of Non-Adaptive Group Testing (NAGT) with runlength constraints on the columns of the test matrix, in the sense that any two 1's must be separated by a run of at least d 0's. We describe and analyze a probabilistic construction of a runlength-constrained scheme in the zero-error and vanishing error settings, and show that the number of tests required by this construction is optimal up to logarithmic factors in the runlength constraint d and the number of defectives k in both cases. Surprisingly, our results show that runlength-constrained NAGT is not more demanding than unconstrained NAGT when d = O(k), and that for almost all choices of d and k it is not more demanding than NAGT with a column Hamming weight constraint only. Towards obtaining runlength-constrained Quantitative NAGT (QNAGT) schemes with good parameters, we also provide lower bounds for this setting and a nearly optimal probabilistic construction of a QNAGT scheme with a column Hamming weight constraint.

show abstract

“…While AMP is clearly fast to run and easy to implement, it also achieves the best algorithmic performance presently known in some of the most prominent inference problems like compressed sensing [67] or the pooled data problem [70].…”

Section: Approximate Message Passing Ampmentioning

confidence: 99%

Large discrete structures : statistical inference, combinatorics and limits

Hahn‐Klimroth¹

View full text Add to dashboard Cite

Studying large discrete systems is of central interest in, non-exclusively, discrete mathematics, computer sciences and statistical physics. The study of phase transitions, e.g. points in the evolution of a large random system in which the behaviour of the system changes drastically, became of interest in the classical field of random graphs, the theory of spin glasses as well as in the analysis of algorithms [78,82, 121]. It turns out that ideas from the statistical physics’ point of view on spin glass systems can be used to study inherently combinatorial problems in discrete mathematics and theoretical computer sciences(for instance, satisfiability) or to analyse phase transitions occurring in inference problems (like the group testing problem) [68, 135, 168]. A mathematical flaw of this approach is that the physical methods only render mathematical conjectures as they are not known to be rigorous. In this thesis, we will discuss the results of six contributions. For instance, we will explore how the theory of diluted mean-field models for spin glasses helps studying random constraint satisfaction problems through the example of the random 2−SAT problem. We will derive a formula for the number of satisfying assignments that a random 2−SAT formula typically possesses [2]. Furthermore, we will discuss how ideas from spin glass models (more precisely, from their planted versions) can be used to facilitate inference in the group testing problem. We will answer all major open questions with respect to non-adaptive group testing if the number of infected individuals scales sublinearly in the population size and draw a complete picture of phase transitions with respect to the complexity and solubility of this inference problem [41, 46]. Subsequently, we study the group testing problem under sparsity constrains and obtain a (not fully understood) phase diagram in which only small regions stay unexplored [88]. In all those cases, we will discover that important results can be achieved if one combines the rich theory of the statistical physics’ approach towards spin glasses and inherent combinatorial properties of the underlying random graph. Furthermore, based on partial results of Coja-Oghlan, Perkins and Skubch [42] and Coja-Oghlan et al. [49], we introduce a consistent limit theory for discrete probability measures akin to the graph limit theory [31, 32, 128] in [47]. This limit theory involves the extensive study of a special variant of the cut-distance and we obtain a continuous version of a very simple algorithm, the pinning operation, which allows to decompose the phase space of an underlying system into parts such that a probability measure, restricted to this decomposition, is close to a product measure under the cut-distance. We will see that this pinning lemma can be used to rigorise predictions, at least in some special cases, based on the physical idea of a Bethe state decomposition when applied to the Boltzmann distribution. Finally, we study sufficient conditions for the existence of perfect matchings, Hamilton cycles and bounded degree trees in randomly perturbed graph models if the underlying deterministic graph is sparse [93].

show abstract

Decoding from pooled data: Phase transitions of message passing

Cited by 9 publications

References 9 publications

Optimal group testing

Optimal group testing

Group Testing with Runlength Constraints for Topological Molecular Storage

Large discrete structures : statistical inference, combinatorics and limits

Contact Info

Product

Resources

About