Toward Super-Polynomial Size Lower Bounds for Depth-Two Threshold Circuits

Chen, Lijie

doi:10.48550/arxiv.1805.10698

Cited by 2 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 ) [Razborov, 1992b, Oliveira, 2015, Chen, 2018. By Theorem 3.1, proving existence of a natural function that cannot be approximated by a network of depth k ≥ 4 would imply that EXP ⊆ TC 0 k−2 and thus solve this open problem.…”

Section: It Is Exponential-time Computablementioning

confidence: 99%

Size and Depth Separation in Approximating Benign Functions with Neural Networks

Vardi,

Reichman,

Pitassi

et al. 2021

Preprint

View full text Add to dashboard Cite

When studying the expressive power of neural networks, a main challenge is to understand how the size and depth of the network affect its ability to approximate real functions. However, not all functions are interesting from a practical viewpoint: functions of interest usually have a polynomially-bounded Lipschitz constant, and can be computed efficiently. We call functions that satisfy these conditions "natural", and explore the benefits of size and depth for approximation of natural functions with ReLU networks. As we show, this problem is more challenging than the corresponding problem for non-natural functions. We give complexity-theoretic barriers to showing depth-lower-bounds: Proving existence of a natural function that cannot be approximated by polynomial-sized networks of depth 4 would settle longstanding open problems in computational complexity. It implies that beyond depth 4 there is a barrier to showing depth-separation for natural functions, even between networks of constant depth and networks of nonconstant depth. We also study size-separation, namely, whether there are natural functions that can be approximated with networks of size O(s(d)), but not with networks of size O(s ′ (d)). We show a complexity-theoretic barrier to proving such results beyond size O(d log 2 (d)), but also show an explicit natural function, that can be approximated with networks of size O(d) and not with networks of size o(d/ log d). For approximation in the L ∞ sense we achieve such separation already between size O(d) and size o(d). Moreover, we show superpolynomial size lower bounds and barriers to such lower bounds, depending on the assumptions on the function. Our size-separation results rely on an analysis of size lower bounds for Boolean functions, which is of independent interest: We show linear size lower bounds for computing explicit Boolean functions with neural networks and threshold circuits.

show abstract

Section: It Is Exponential-time Computablementioning

confidence: 99%

Size and Depth Separation in Approximating Benign Functions with Neural Networks

Vardi,

Reichman,

Pitassi

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, there have been a lot of results connecting BCP or (1 + o(1))-BCP to other problems (see [50,15,16,17]). Now such connections can be extended to CP as well.…”

Section: :4mentioning

confidence: 99%

On Closest Pair in Euclidean Metric: Monochromatic is as Hard as Bichromatic

Karthik

Manurangsi

2020

Combinatorica

View full text Add to dashboard Cite

Given a set of n points in R d , the (monochromatic) Closest Pair problem asks to find a pair of distinct points in the set that are closest in the p -metric. Closest Pair is a fundamental problem in Computational Geometry and understanding its fine-grained complexity in the Euclidean metric when d = ω(log n) was raised as an open question in recent works (Abboud-Rubinstein-Williams [FOCS'17], Williams [SODA'18], ).In this paper, we show that for every p ∈ R ≥1 ∪ {0}, under the Strong Exponential Time Hypothesis (SETH), for every ε > 0, the following holds:No algorithm running in time O(n 2−ε ) can solve the Closest Pair problem in d = (log n) Ωε(1) dimensions in the p -metric.ThereIn particular, our first result is shown by establishing the computational equivalence of the bichromatic Closest Pair problem and the (monochromatic) Closest Pair problem (up to n ε factor in the running time) for d = (log n) Ωε(1) dimensions.Additionally, under SETH, we rule out nearly-polynomial factor approximation algorithms running in subquadratic time for the (monochromatic) Maximum Inner Product problem where we are given a set of n points in n o(1) -dimensional Euclidean space and are required to find a pair of distinct points in the set that maximize the inner product.At the heart of all our proofs is the construction of a dense bipartite graph with low contact dimension, i.e., we construct a balanced bipartite graph on n vertices with n 2−ε edges whose vertices can be realized as points in a (log n) Ωε(1) -dimensional Euclidean space such that every pair of vertices which have an edge in the graph are at distance exactly 1 and every other pair of vertices are at distance greater than 1. This graph construction is inspired by the construction of locally dense codes introduced by Dumer-Miccancio-Sudan [IEEE Trans. Inf. Theory'03].

show abstract

Toward Super-Polynomial Size Lower Bounds for Depth-Two Threshold Circuits

Cited by 2 publications

References 26 publications

Size and Depth Separation in Approximating Benign Functions with Neural Networks

Size and Depth Separation in Approximating Benign Functions with Neural Networks

On Closest Pair in Euclidean Metric: Monochromatic is as Hard as Bichromatic

Contact Info

Product

Resources

About