Multiclass classification by Min–Max ECOC with Hamming distance optimization

Abstract

Two questions often arise in the field of the ensemble in multiclass classification problems, (i) how to combine base classifiers and (ii) how to design possible binary classifiers. Error-correcting output codes (ECOC) methods answer these questions, but they focused on only the general goodness of the classifier. The main purpose of our research was to strengthen the bottleneck of the ensemble method, i.e., to minimize the largest values of two types of error ratios in the deep neural network-based classifier. The research was theoretical and experimental, the proposed Min–Max ECOC method suggests a theoretically proven optimal solution, which was verified by experiments on image datasets. The optimal solution was based on the maximization of the lowest value in the Hamming matrix coming from the ECOC matrix. The largest ECOC matrix, the so-called full matrix is always a Min–Max ECOC matrix, but smaller matrices generally do not reach the optimal Hamming distance value, and a recursive construction algorithm was proposed to get closer to it. It is not easy to calculate optimal values for large ECOC matrices, but an interval with upper and lower limits was constructed by two theorems, and they were proved. Convolutional Neural Networks with Min–Max ECOC matrix were tested on four real datasets and compared with OVA (one versus all) and variants of ECOC methods in terms of known and two new indicators. The experimental results show that the suggested method surpasses the others, thus our method is promising in the ensemble learning literature.

Greedy control group selection for multi-explanatory multi-output regression problem

“…The multi-output learning can be divided into two larger fields, multi-label learning [41] (multi-label classification [35], in which some of the solutions also can be considered as multi-output problem, because these kind of solutions reduces the multi-class problems to a family of binary ones, e.g. by one-versus-all, by all-versus-all [15], or by ECOC -Error-Correcting Output Codes [34]) and multi-output regression [40]. Despite the recent advances, multi-output regression is still an open challenge for developing a low-cost and highly accurate algorithm [23], for underlying relationships between input features and output targets, and for exploring inter-target dependencies [30] [39].…”

Multi-target and control group selection for multi-explanatory multi-output regression problem

“…The multi-output learning can be divided into two larger fields, multi-label learning [39] (multi-label classification [33], in which some of the solutions also can be considered as multi-output problem, because these kind of solutions reduces the multi-class problems to a family of binary ones, e.g. by one-versus-all, by all-versus-all [14], or by ECOC -Error-Correcting Output Codes [32]) and multi-target regression [38]. Despite the recent advances, multi-target regression is still an open challenge for developing a low-cost and highly accurate algorithm [22], for underlying relationships between input features and output targets, and for exploring inter-target dependencies [29] [37].…”

Fault diagnosis of power transformers based on t-SNE and ECOC-TEWSO-SVM

“…ECOC combines machine learning methods with errorcorrecting output coding to achieve multi-class recognition and error correction of models during the transmission process. 18 ECOC mainly uses binary encoding to transform n classifications into n(n − 1)/2 binary classification problems for solving. Specifically for the problem presented in this paper, the 6-class classification is transformed into 15 binary classification problems for solving, as shown in Table VIII, where 1 and −1 represent positive and negative sample problems, respectively, and 0 represents ignoring this problem.…”