A Boosting Approach to Multiview Classification with Cooperation

Abstract

Abstract. In many fields, such as bioinformatics or multimedia, data may be described using different sets of features (or views) which carry either global or local information. Some learning tasks make use of these several views in order to improve overall predictive power of classifiers through fusion-based methods. Usually, these approaches rely on a weighted combination of classifiers (or selected descriptions), where classifiers are learned independently. One drawback of these methods is that the classifier learned on one view does not communicate its failures within the other views. This paper deals with a novel approach to integrate multiview information. The proposed algorithm, named Mumbo, is based on boosting. Within the boosting scheme, Mumbo maintains one distribution of examples on each view, and at each round, it learns one weak classifier on each view. Within a view, the distribution of examples evolves both with the ability of the dedicated classifier to deal with examples of the corresponding features space, and with the ability of classifiers in other views to process the same examples within their own description spaces. Hence, the principle is to slightly remove the hard examples from the learning space of one view, while their weights get higher in the other views. This way, we expect that examples are urged to be processed by the most appropriate views, when possible. At the end of the iterative learning process, a final classifier is computed by a weighted combination of selected weak classifiers.This paper provides the Mumbo algorithm in a multiclass and multiview setting, based on recent theoretical advances in boosting. The boosting properties of Mumbo are proved, as well as some results on its generalization capabilities. Several experimental results are reported which point out that complementary views may actually cooperate under some assumptions.

Forward Stagewise Additive Model for Collaborative Multiview Boosting

“…Mumbo is an elegant example of multiview assisted boosting algorithm [30], [31]. The fundamental idea of Mumbo is to remove an arduous example from view space of weak learners and simultaneously increase weight of that example in view space of strong learners.…”

“…1) Comparison of Generalization Accuracy Rates: In this section we evaluate our proposed boosting algorithm on the [18] 0.83 0.78 Zhang et al [20] 0.86 0.82 Co-AdaBoost [27] 5 0.86 0.84 2-Boost [26] 0.86 0.81 AdaBoost.Group [29] 0.83 0.81 Mumbo [30] 0.88 0.87 MA-AdaBoost [1] 0 benchmark UCI datasets which comprise of real world data pertaining to financial credit rating, medical diagnosis, game playing etc. The details of the eleven datasets chosen for simulation is shown in Table VII.…”

Internet cross-media retrieval based on deep learning

“…An active transfer learning method is proposed to correlate the features of different scenes [17]. Based on boosting, a weak classifier is learned in each mode and the integrated classifier is produced by weighted combination [18].…”

Multiview Learning of Weighted Majority Vote by Bregman Divergence Minimization

“…Amini et al Amini et al [2009] derived a generalization error bound for classifiers learned on multiview examples and identified situations where it is more interesting to use all views to learn a uniformly weighted majority vote classifier instead of single view learning. Koço et al Koço and Capponi [2011] proposed a Boosting-based strategy that maintains a different distribution of examples with respect to each view. For a given view, the corresponding distribution is updated based on view-specific weak classifiers from that view and all the other views with the idea of using all the view-specific distributions to weight hard examples for the next iteration.…”