Solving for multi-class using orthogonal coding matrices

Abstract

A common method of generalizing binary to multi-class classification is the error correcting code (ECC). ECCs may be optimized in a number of ways, for instance by making them orthogonal. Here we test two types of orthogonal ECCs on seven different datasets using three types of binary classifier and compare them with three other multi-class methods: 1 vs. 1, one-versus-the-rest and random ECCs. The first type of orthogonal ECC, in which the codes contain no zeros, admits a fast and simple method of solving for the probabilities. Orthogonal ECCs are always more accurate than random ECCs as predicted by recent literature. Improvments in uncertainty coefficient (U.C.) range between 0.4-17.5% (0.004-0.139, absolute), while improvements in Brier score between 0.7-10.7%. Unfortunately, orthogonal ECCs are rarely more accurate than 1 vs. 1. Disparities are worst when the methods are paired with logistic regression, with orthogonal ECCs never beating 1 vs. 1. When the methods are paired with SVM, the losses are less significant, peaking at 1.5%, relative, 0.011 absolute in uncertainty coefficient and 6.5% in Brier scores. Orthogonal ECCs are always P. Mills (2019) SN Applied Sciences 1(11):1451 2 the fastest of the five multi-class methods when paired with linear classifiers. When paired with a piecewise linear classifier, whose classification speed does not depend on the number of training samples, classifications using orthogonal ECCs were always more accurate than the the remaining three methods and also faster than 1 vs. 1. Losses against 1 vs. 1 here were higher, peaking at 1.9% (0.017, absolute), in U.C. and 39% in Brier score. Gains in speed ranged between 1.1% and over 100%. Whether the speed increase is worth the penalty in accuracy will depend on the application.

Multi-Class Counterfactual Explanations using Support Vector Data Description

“…In these cases, binary classifiers must be adapted to handle multiple outputs. Therefore, we can distinguish two types of multi-class classification techniques [9]: one-vsone and one-vs-rest. In one-vs-one techniques the problem is divided into m(m−1) 2 binary classifiers, where m is the number of classes and each binary classifier predicts a class label.…”

A novel ensemble approach based on MCC and MCDM methods for prioritizing tweets mentioning urban issues in smart city

“…A continuous stream of tweets makes the task of manually distinguishing the critical issues from the noncritical ones very strenuous and imprecise. So here in this study, we implement the automatic classification of the complaints into three classes, like “moderate,” “urgent” and “immediate” using the multi-class classification (MCC) algorithm (Mills, 2018). Instead of the traditional “urgent” and “non-urgent” binary classification, the proposed approach splits the issues into three classes for more precise categorization as manual efforts would be required to identify issues from “urgent” tweets, which need immediate action.…”

Self-Dual Codes, Symmetric Matrices, and Eigenvectors

“…The theory of error-correcting code, which was born with the invention of computers, has been an interesting topic of mathematics as well as industry, such as satellites, CD players, and cellular phones. Recently, with the advent of machine learning and artificial intelligence, there have been some studies on the relationship between error-correcting codes and these fields [2], [22], [30], [31]. Especially, selfdual codes have been an important class of linear codes for both practical and theoretical reasons and have received an enormous research effort since the beginning of coding theory.…”