MICE: Multi-Layer Multi-Model Images Classifier Ensemble

Abstract

In this paper, a new type of fast deep learning (DL) network for handwriting recognition is proposed. In contrast to the existing DL networks the proposed approach has clearly interpretable structure that is entirely data-driven and free from user-or problem-specific assumptions. It is entirely parallelizable and very efficient. First, same fundamental image transformation techniques (rotation and scaling) that are used by other existing DL methods are used to improve the generalization. The commonly used descriptors are then used to extract the global features from the training set and based on them a bank/ensemble of zero order AnYa type fuzzy rule-based (FRB) models is built through the recently introduced Autonomous Learning Multiple Model (ALMMo) method working in parallel. The final decision about the winning class label is made by a committee on the basis of the fuzzy mixture of the trained ALMMo-0 models (where "0" stands for 0 order meaning that the consequent represent a class label, a singleton, not a regression model as in the first order). The training of the proposed MICE system is very efficient and highly parallelizable. It significantly outperforms the best known methods in terms of time and is on par in terms of precision/accuracy. Critically, it offers a high level of interpretability, transparency of the classification model, full repeatability (unlike the methods that use probabilistic elements) of the results. Moreover, it allows an evolving scenario whereby the data is provided in an incremental, online manner and the system structure is being developed in parallel with the classification which opens opportunities for online and real-time applications (on a sample by sample basis). Numerical examples from the well-known handwritten digits recognition problem (MNIST) were used and the results demonstrated the very high repeatable performance after a very short training process which is in addition to the high level of interpretability, transparency.

Autonomous Learning Multimodel Systems From Data Streams

“…The most important point is that the proposed ALMMo classifier is entirely data-driven and is free from unrealistic assumptions, restrictions or problem-or userspecific prior knowledge. In other studies, we also demonstrated that ALMMo classifiers are able to achieve comparable or even better classification results against the state-of-theart approaches in more complicated problems including, but not limited to, handwriting digital recognition, remote sensing image classification without the need of predefining (problemor user-specific) parameters and thresholds [40].…”

Evolving Ensemble Fuzzy Classifier

“…This work can be classified as an ensemble in a strict sense where data are distributed in a number of computing nodes. In [70], an ensemble of deep learning classifiers was designed for handwriting recognition and adopted the concept of data parallerization as with [69]. The all-pair classifier in [50] can be also grouped as an ensemble approach.…”

“…Motivated by significant progress in real-time data collection and capture, the notion of EIS has gained popularity in the community because it has been shown effective in addressing lifelong learning situation and non-stationary environments. Several extensions and variations of EIS have been put forward in the literature [39], [40], [67]- [70]. An evolving version of Vector quantization was designed in [41] and is algorithmic backbone of FLEXFIS [42], which was later extended to a more robust version including rule merging in [43], generalized rules and an incremental feature weighting mechanism in [44].…”

“…Notwithstanding that the EIS has been well-established in the literature, it still deserves in-depth investigation because of at least three reasons 1) vast majority of EIS is constructed in the single model framework having low diversity. The ensemble learning concept is well-known for its powerful generalization power because it address the bias-and-variance better and produces a model with high diversity covering a rich data region; 2) The use of evolving base classifier in the ensemble structure has been initiated in [14], [15], [50], [69], [70] but it relies on a static ensemble structure which is predetermined during the training process; 3) Existing EISs are categorized as a passive approach in handling concept drift because changing data distributions are overcome by continuously adapting a classifier. It lacks of capability to signal the presence of concept drift and to identify the type of drift.…”

A Massively Parallel Deep Rule-Based Ensemble Classifier for Remote Sensing Scenes

“…In this Letter, a new approach based on the ensemble of the recently introduced deep (fuzzy) rule-based (DRB) classifiers [11], [12] is proposed for remote sensing scene classification. The DRB classifier employs a massively parallel set of 0-order fuzzy rules as a learning engine [13].…”