Anytime Classification Using the Nearest Neighbor Algorithm with Applications to Stream Mining

Abstract

For many real world problems we must perform classification under widely varying amounts of computational resources. For example, if asked to classify an instance taken from a bursty stream, we may have from milliseconds to minutes to return a class prediction. For such problems an anytime algorithm may be especially useful.In this work we show how we can convert the ubiquitous nearest neighbor classifier into an anytime algorithm that can produce an instant classification, or if given the luxury of additional time, can utilize the extra time to increase classification accuracy. We demonstrate the utility of our approach with a comprehensive set of experiments on data from diverse domains.

Exact Primitives for Time Series Data Mining

“…There has been recent interest in converting classic batch data mining algorithms to anytime versions [104]. In some cases this is trivial, for example we can frame the nearest-neighbor classification algorithm as an anytime algorithm simply by conducting a sequential search for the unlabeled items nearest neighbor in the labeled dataset [104].…”

“…In some cases this is trivial, for example we can frame the nearest-neighbor classification algorithm as an anytime algorithm simply by conducting a sequential search for the unlabeled items nearest neighbor in the labeled dataset [104]. If the algorithm is interrupted before completing the full search, then the label of the best-so-far nearest neighbor is returned as the class label.…”

“…The ordering is typically temporal; however other kinds of data such as color distributions [41], shapes [104] and spectrographs [117] also have a well defined ordering and can fruitfully be considered "time series" for the purpose of indexing and mining. It is possible there could be variable time spacing between successive points in the series.…”

“…This simple idea can be enhanced by sorting the labeled instances such that the most useful instances are seen early in the sequential search. In all work that we are aware of, "most useful" is determined by some estimate of how often each instance is used to correctly predict, as opposed to incorrectly predict, unknown instances [104][111].…”

Dictionary-Based Compression for Long Time-Series Similarity

“…The dictionary that gives the best score confers its label to the unlabeled time-series. This method is the nearest neighbor classification technique [29], [33], [52], where an unlabeled series is assigned its class label of its nearest neighbor in the training set. It has been argued that nearest neighbor classification is rather robust in its ability to resist the effect of noise.…”