Cross-dataset clustering

Dagan, Ido; Marx, Zvika; Shamir, Eli

doi:10.3115/1118853.1118862

Cited by 6 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The communication complexity of estimating one-dimensional Gaussian correlations is established in [19] and that of independence testing over discrete alphabet in the large sample regime is characterized in [9]. The tradeoff between communication complexity and sample complexity for detecting pairwise correlations is studied in [15]. A related line of recent work considers composite hypothesis testing under communication, privacy, and shared randomness constraints [1]- [4].…”

Section: Introductionmentioning

confidence: 99%

Communication Complexity of Distributed High Dimensional Correlation Testing

Sahasranand

Tyagi

2021

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We consider a two-party distributed hypothesis testing problem for correlated Gaussian random variables. For a d-dimensional random vector X and a scalar random variable Y , where X and Y are jointly Gaussian with an unknown correlation vector ρ, parties P1 and P2 observe independent copies of X and Y , respectively. The parties seek to test if their observations are correlated or not, namely they seek to test if ρ2 exceeds ς or is it 0. To that end, they communicate interactively and declare the test output. We show that roughly order d/ς 2 bits of communication are sufficient and necessary for resolving the distributed correlation testing problem above. Furthermore, we establish a lower bound of roughly d 2 /ς 2 bits for the communication needed for distributed estimation of ρ, implying that distributed correlation testing requires less communication than distributed estimation. Both our lower bounds for testing and estimation hold for an arbitrary d and interactive communication with shared randomness, while our distributed test requires only one-way communication with shared randomness. For the one-dimensional case, with one-way communication and with probability of one of the error-types fixed, our bounds are more refined in the dependence on the other error-type and are tight even in the constant.

show abstract

Section: Introductionmentioning

confidence: 99%

Communication Complexity of Distributed High Dimensional Correlation Testing

Sahasranand

Tyagi

2021

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

Corpus-based Learning of Analogies and Semantic Relations

2005

View full text Add to dashboard Cite

Abstract. We present an algorithm for learning from unlabeled text, based on the Vector Space Model (VSM) of information retrieval, that can solve verbal analogy questions of the kind found in the SAT college entrance exam. A verbal analogy has the form A:B::C:D, meaning "A is to B as C is to D"; for example, mason:stone::carpenter:wood. SAT analogy questions provide a word pair, A:B, and the problem is to select the most analogous word pair, C:D, from a set of five choices. The VSM algorithm correctly answers 47% of a collection of 374 college-level analogy questions (random guessing would yield 20% correct; the average college-bound senior high school student answers about 57% correctly). We motivate this research by applying it to a difficult problem in natural language processing, determining semantic relations in noun-modifier pairs. The problem is to classify a noun-modifier pair, such as "laser printer", according to the semantic relation between the noun (printer) and the modifier (laser). We use a supervised nearest-neighbour algorithm that assigns a class to a given noun-modifier pair by finding the most analogous noun-modifier pair in the training data. With 30 classes of semantic relations, on a collection of 600 labeled noun-modifier pairs, the learning algorithm attains an F value of 26.5% (random guessing: 3.3%). With 5 classes of semantic relations, the F value is 43.2% (random: 20%). The performance is state-of-the-art for both verbal analogies and noun-modifier relations.

show abstract

Cross-partition clustering: revealing corresponding themes across related datasets

Marx

Dagan

Shamir

2011

Journal of Experimental & Theoretical Artificial Intelligen

Self Cite

View full text Add to dashboard Cite

This article studies the task of discovering correspondences across related domains based on real-world data collections. We address this task through a designated extension of distributional data-clustering methods. The method is empirically demonstrated on synthetic data as well as on texts addressing different religions, where the goal is to identify commonalities shared by all religions. This article generalises and demonstrates the empirical improvement relative to our previous studies on this subject, as well as to other comparable methods.

show abstract

Cross-dataset clustering

Cited by 6 publications

References 7 publications

Communication Complexity of Distributed High Dimensional Correlation Testing

Communication Complexity of Distributed High Dimensional Correlation Testing

Corpus-based Learning of Analogies and Semantic Relations

Cross-partition clustering: revealing corresponding themes across related datasets

Contact Info

Product

Resources

About