Comparative Classifier Aggregation

Finding the Most Probable Ranking of Objects with Probabilistic Pairwise Preferences

“…There were sets from 5 to 40 objects in the artificial dataset. The second dataset was based on the real probabilities being extracted from the Microsoft On-Line Handwriting Recognizer [1]. The recognizer returns a list of up to 30 possible alternatives for an ink word, and for all pairs of alternatives the recognizer produces pairwise preference probabilities.…”

“…As an example, consider an on-line handwriting word recognizer [1]. The result of recognition is the list of possible alternatives with some additional information provided for each hypothesis (such as the raw scores from each of the differenent classifiers inside the recognizer, the confidences for every letter in the word, the unigram frequency of the word in the vocabulary, etc), as discussed in [1].…”

“…The result of recognition is the list of possible alternatives with some additional information provided for each hypothesis (such as the raw scores from each of the differenent classifiers inside the recognizer, the confidences for every letter in the word, the unigram frequency of the word in the vocabulary, etc), as discussed in [1]. We may want to sort this list according to the probability of each answer being correct, so that even if the first hypothesis is incorrect, the correct one will likely be close in the list, and it will be easy for a user to make corrections by simply choosing the right hypothesis from the list.…”