Optimal Cryptographic Hardness of Learning Monotone Functions

Dachman-Soled, Dana; Lee, Homin K.; Malkin, Tal; Servedio, Rocco A.; Wan, Andrew; Wee, Hoeteck

doi:10.1007/978-3-540-70575-8_4

Cited by 5 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From a foundational perspective, submodular functions are a powerful, broad class of important functions, so studying their learnability allows us to understand their structure in a new way. To draw a parallel to the Boolean-valued case, a class of comparable breadth would be the class of monotone Boolean functions; the learnability of such functions has been intensively studied [4,5]. From an applications perspective, algorithms for learning submodular functions may be useful in some of the applications where these functions arise.…”

mentioning

confidence: 99%

Learning Submodular Functions

Balcan

Harvey

2012

Machine Learning and Knowledge Discovery in Databases

188

View full text Add to dashboard Cite

Abstract. Submodular functions are discrete functions that model laws of diminishing returns and enjoy numerous algorithmic applications that have been used in many areas, including combinatorial optimization, machine learning, and economics. In this work we use a learning theoretic angle for studying submodular functions. We provide algorithms for learning submodular functions, as well as lower bounds on their learnability. In doing so, we uncover several novel structural results revealing both extremal properties as well as regularities of submodular functions, of interest to many areas.Submodular functions are a discrete analog of convex functions that enjoy numerous applications and have structural properties that can be exploited algorithmically. They arise naturally in the study of graphs, matroids, covering problems, facility location problems, etc., and they have been extensively studied in operations research and combinatorial optimization for many years [8]. More recently submodular functions have become key concepts both in the machine learning and algorithmic game theory communities. For example, submodular functions have been used to model bidders' valuation functions in combinatorial auctions [12,6,3,14], and for solving feature selection problems in graphical models [11] or for solving various clustering problems [13]. In fact, submodularity has been the topic of several tutorials and workshops at recent major conferences in machine learning [1,9,10,2].Despite the increased interest on submodularity in machine learning, little is known about the topic from a learning theory perspective. In this work, we provide a statistical and computational theory of learning submodular functions in a distributional learning setting.Our study has multiple motivations. From a foundational perspective, submodular functions are a powerful, broad class of important functions, so studying their learnability allows us to understand their structure in a new way. To draw a parallel to the Boolean-valued case, a class of comparable breadth would be the class of monotone Boolean functions; the learnability of such functions has been intensively studied [4,5]. From an applications perspective, algorithms for learning submodular functions may be useful in some of the applications where these functions arise. For example, in the context of algorithmic game theory This note summarizes several results in the paper "Learning Submodular Functions", by Maria Florina Balcan and Nicholas Harvey, which appeared The 43rd ACM Symposium on Theory of Computing (STOC) 2011.

show abstract

mentioning

confidence: 99%

Learning Submodular Functions

Balcan

Harvey

2012

Machine Learning and Knowledge Discovery in Databases

188

View full text Add to dashboard Cite

show abstract

“…On the other hand, the central role of the uniform distribution in computational complexity and cryptography relates learning under the uniform distribution to key themes in theoretical computer science including de-randomization, hardness and cryptography, e.g. (Kharitonov, 1993, Naor and Reingold, 2004, Dachman-Soled et al, 2008.…”

Section: Learning Under the Uniform Distributionmentioning

confidence: 99%

MCMC Learning

Kanade,

Mossel

2013

Preprint

View full text Add to dashboard Cite

The theory of learning under the uniform distribution is rich and deep, with connections to cryptography, computational complexity, and the analysis of boolean functions to name a few areas. This theory however is very limited due to the fact that the uniform distribution and the corresponding Fourier basis are rarely encountered as a statistical model.A family of distributions that vastly generalizes the uniform distribution on the Boolean cube is that of distributions represented by Markov Random Fields (MRF). Markov Random Fields are one of the main tools for modeling high dimensional data in many areas of statistics and machine learning.In this paper we initiate the investigation of extending central ideas, methods and algorithms from the theory of learning under the uniform distribution to the setup of learning concepts given examples from MRF distributions. In particular, our results establish a novel connection between properties of MCMC sampling of MRFs and learning under the MRF distribution.

show abstract

Section: Introductionmentioning

confidence: 99%

Learning submodular functions

Balcan

Harvey

2011

Proceedings of the Forty-Third Annual ACM Symposium on Theory of Computing

View full text Add to dashboard Cite

This paper considers the problem of learning submodular functions. A problem instance consists of a distribution on {0, 1}n and a real-valued function on {0, 1} n that is non-negative, monotone and submodular. We are given poly(n) samples from this distribution, along with the values of the function at those sample points. The task is to approximate the value of the function to within a multiplicative factor at subsequent sample points drawn from the distribution, with sufficiently high probability. We prove several results for this problem.• There is an algorithm that, for any distribution, approximates any such function to within a factor of √ n on a set of measure 1 − ϵ.• There is a distribution and a family of functions such that no algorithm can approximate those functions to within a factor of Õ(n 1/3 ) on a set of measure 1/2 + ϵ.• If the function is a matroid rank function and the distribution is a product distribution then there is an algorithm that approximates it to within a factor O (log(1/ϵ)) on a set of measure 1 − ϵ.Our study involves a twist on the usual learning theory models and uncovers some interesting extremal properties of submodular functions.

show abstract

Optimal Cryptographic Hardness of Learning Monotone Functions

Cited by 5 publications

References 27 publications

Learning Submodular Functions

Learning Submodular Functions

MCMC Learning

Learning submodular functions

Contact Info

Product

Resources

About