On multiple-instance learning of halfspaces

Diochnos, Dimitrios I.; Sloan, Robert H.; Turán, Gy

doi:10.1016/j.ipl.2012.08.017

Cited by 3 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Theorem 1 is in line with previous complexity results for MI classification via hyperplanes (Kundakcioglu et al 2010;Diochnos et al 2012). For clarity, we include the theorem below, expressed in terms of our formalism:…”

Section: Can All Three Properties Be Satisfied?supporting

confidence: 75%

“…The proof of Theorem 2 (Diochnos et al 2012) reduces a 3-SAT instance to an instance of MI-CONSIS such that there is a strongly consistent hyperplane if the 3-SAT formula is satisfiable and no consistent (in the usual sense) hyperplane if the formula is not satisfiable. Thus, the proof works for either notion of consistency, though the distinction is not made in the original work.…”

Section: Can All Three Properties Be Satisfied?mentioning

confidence: 99%

See 1 more Smart Citation

A theoretical and empirical analysis of support vector machine methods for multiple-instance classification

Doran

Ray

2013

Mach Learn

View full text Add to dashboard Cite

The standard support vector machine (SVM) formulation, widely used for supervised learning, possesses several intuitive and desirable properties. In particular, it is convex and assigns zero loss to solutions if, and only if, they correspond to consistent classifying hyperplanes with some nonzero margin. The traditional SVM formulation has been heuristically extended to multiple-instance (MI) classification in various ways. In this work, we analyze several such algorithms and observe that all MI techniques lack at least one of the desirable properties above. Further, we show that this tradeoff is fundamental, stems from the topological properties of consistent classifying hyperplanes for MI data, and is related to the computational complexity of learning MI hyperplanes. We then study the empirical consequences of this three-way tradeoff in MI classification using a large group of algorithms and datasets. We find that the experimental observations generally support our theoretical results, and properties such as the labeling task (instance versus bag labeling) influence the effects of different tradeoffs.

show abstract

Section: Can All Three Properties Be Satisfied?supporting

confidence: 75%

Section: Can All Three Properties Be Satisfied?mentioning

confidence: 99%

A theoretical and empirical analysis of support vector machine methods for multiple-instance classification

Doran

Ray

2013

Mach Learn

View full text Add to dashboard Cite

show abstract

“…From the perspective of computational complexity, Auer et al reduce learning DNF formulae to learning APRs from MI data (1998), showing that efficiently PAC-learning these MI instance concepts is impossible (unless NP = RP) when arbitrary distributions over r-tuples are allowed. Similarly, finding classifying hyperplanes for MI data has been shown to be NP-complete (Diochnos, Sloan, and Turán 2012;Kundakcioglu, Seref, and Pardalos 2010). However, all of these reductions rely on generating bags such that certain negative instances only appear in positive bags.…”

Section: Relation To Previous Workmentioning

confidence: 99%

Learning Instance Concepts from Multiple-Instance Data with Bags as Distributions

Doran

Ray

2014

AAAI

View full text Add to dashboard Cite

We analyze and evaluate a generative process for multiple-instance learning (MIL) in which bags are distributions over instances. We show that our generative process contains as special cases generative models explored in prior work, while excluding scenarios known to be hard for MIL. Further, under the mild assumption that every negative instance is observed with nonzero probability in some negative bag, we show that it is possible to learn concepts that accurately label instances from MI data in this setting. Finally, we show that standard supervised approaches can learn concepts with low area-under-ROC error from MI data in this setting. We validate this surprising result with experiments using several synthetic and real-world MI datasets that have been annotated with instance labels.

show abstract

“…It follows from recent results of Sabato and Tishby [20] that an algorithm for "Minimum One-sided Disagreement" can be transformed (without much loss of efficiency) into an algorithm that is successful in the hard version of the MIL model. The learning problem for the hard version of the MIL model is known to be NP-hard for axis-parallel hyper-rectangles of variable dimension (according to a result in [5]) 1 and for Euclidean halfspaces of variable dimension (according to results in [10]. On the positive side, our algorithms for the classes "Unions of k Intervals", "Axis-parallel Rectangles", T 2,k and TREE(2, n, 2, k) can be used as subroutines to solve the hard version of the corresponding MIL problem.…”

Section: Learning From Multiple-instance Examples (Mil)mentioning

confidence: 99%

PAC-learning in the presence of one-sided classification noise

Simon

2012

Ann Math Artif Intell

View full text Add to dashboard Cite

We derive an upper and a lower bound on the sample size needed for PAClearning a concept class in the presence of one-sided classification noise. The upper bound is achieved by the strategy "Minimum One-sided Disagreement". It matches the lower bound (which holds for any learning strategy) up to a logarithmic factor. Although "Minimum One-sided Disagreement" often leads to NP-hard combinatorial problems, we show that it can be implemented quite efficiently for some simple concept classes like, for example, unions of intervals, axis-parallel rectangles, and TREE(2, n, 2, k) which is a broad subclass of 2-level decision trees. For the first class, there is an easy algorithm with time bound O(m log m). For the second-one (resp. the third-one), we design an algorithm that applies the well-known UNION-FIND data structure and has an almost quadratic time bound (resp. time bound O(n 2 m log m)).

show abstract

On multiple-instance learning of halfspaces

Cited by 3 publications

References 9 publications

A theoretical and empirical analysis of support vector machine methods for multiple-instance classification

A theoretical and empirical analysis of support vector machine methods for multiple-instance classification

Learning Instance Concepts from Multiple-Instance Data with Bags as Distributions

PAC-learning in the presence of one-sided classification noise

Contact Info

Product

Resources

About