Agnostic Learning of Geometric Patterns

Abstract

) discussed how the problem of recognizing a landmark from a one-dimensional visual image might be mapped to that of learning a onedimensional geometric pattern and gave a PAC algorithm to learn that class. In this paper, we present an efficient online agnostic learning algorithm for learning the class of constant-dimensional geometric patterns. Our algorithm can tolerate both classification and attribute noise. By working in higher dimensional spaces we can represent more features from the visual image in the geometric pattern. Our mapping of the data to a geometric pattern and, hence, our learning algorithm are applicable to any data representable as a constant-dimensional array of values, e.g., sonar data, temporal difference information, amplitudes of a waveform, or other pattern recognition data. To our knowledge, these classes of patterns are more complex than any class of geometric patterns previously studied. Also, our results are easily adapted to learn the union of fixed-dimensional boxes from multiple-instance examples. Finally, our algorithms are tolerant of concept shift, where the target concept that labels the examples can change over time. Academic Press

Multi-instance clustering with applications to multi-instance prediction

“…They also presented a theoretical algorithm with smaller sample complexity than that of the algorithm of Auer et al [9]. Goldman et al [12] presented an efficient on-line agnostic multi-instance learning algorithm for learning the class of constant-dimension geometric patterns, which tolerated both noise and concept shift. Later, this algorithm was extended so that it could deal with real-valued output [13].…”

“…StdMIP techniques have already been applied to diverse applications including content-based image retrieval [34][35][36][37][38], scene classification [16,39], stock selection [14], landmark matching [12,13], computer security [18], subgoal discovery [40], web mining [41], etc.…”

“…Based on this, the real-valued label RL Xi ∈ [0, 1] for molecule X i is the ratio of E Xi to the maximum possible binding energy E max : (12) where E max = − · n k=1 s k . Thresholding at 1/2 converts RL Xi to a binary label BL Xi .…”

A Boosting Approach to Multiple Instance Learning

“…Whereas [3] uses a single rectangle, which works quite well under some assumptions on the distribution of the data, in [6] the authors try to grow axis-parallel rectangles containing at least one instance from each positive bag and no instance from a negative bag. Scott et al [19] adapt and improve an approach of Goldman et al [8] to learn d-dimensional patterns. The latter uses as hypotheses weighted combinations of hyper-rectangles over a discretized feature space, where the weights are learned with Winnow [12].…”

Improved MCMC sampling methods for estimating weighted sums in Winnow with application to DNF learning

“…These results are critical in applying MCMC methods to other applications of MWU algorithms with exponentially large feature spaces. For example, the Winnow-based algorithm of Tao and Scott (2004) (adapted from Goldman et al 2001 for learning concepts from a generalization of the multiple-instance model, Dietterich et al 1997) is efficient for low dimensions, but does not scale well. It is possible that Chawla et al's MCMC-based approach will be very useful to make this algorithm (and others) more scalable, but first a thorough empirical analysis of the sampling method is required.…”