Classification-based objective functions

Rimer, M.E.; Martinez, Tony

doi:10.1007/s10994-006-6266-6

Cited by 23 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Classification-based (CB) error functions [109] heuristically seek to directly minimize classification error by backpropagating network error only on misclassified patterns. In so doing, they perform relatively minimal updates to network parameters in order to discourage premature weight saturation and overfitting.…”

Section: Criterion Functionsmentioning

confidence: 99%

Fundamentals of Machine Learning

Swamy²

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

Learning is a fundamental capability of neural networks. Learning rules are algorithms for finding suitable weights W and/or other network parameters. Learning of a neural network can be viewed as a nonlinear optimization problem for finding a set of network parameters that minimize the cost function for given examples. This kind of parameter estimation is also called a learning or training algorithm.Neural networks are usually trained by epoch. An epoch is a complete run when all the training examples are presented to the network and are processed using the learning algorithm only once. After learning, a neural network represents a complex relationship, and possesses the ability for generalization. To control a learning process, a criterion is defined to decide the time for terminating the process. The complexity of an algorithm is usually denoted as O(m), indicating that the order of number of floating-point operations is m.Learning methods are conventionally divided into supervised, unsupervised, and reinforcement learning; these schemes are illustrated in Fig. 2.1. x p and y p are the input and output of the pth pattern in the training set,ŷ p is the neural network output for the pth input, and E is an error function. From a statistical viewpoint, unsupervised learning learns the pdf of the training set, p(x), while supervised learning learns about the pdf of p( y|x). Supervised learning is widely used in classification, approximation, control, modeling and identification, signal processing, and optimization. Unsupervised learning schemes are mainly used for clustering, vector quantization, feature extraction, signal coding, and data analysis. Reinforcement learning is usually used in control and artificial intelligence.In logic and statistical inference, transduction is reasoning from observed, specific (training) cases to specific (test) cases. In contrast, induction is reasoning from observed training cases to general rules, which are then applied to the test cases. Machine learning falls into two broad classes: inductive learning or transductive learning. Inductive learning pursues the standard goal in machine learning, which is to accurately classify the entire input space. In contrast, transductive learning focuses

show abstract

Section: Criterion Functionsmentioning

confidence: 99%

Fundamentals of Machine Learning

Swamy²

2013

Neural Networks and Statistical Learning

View full text Add to dashboard Cite

show abstract

“…For regression tasks, in which the objective is to approximate the function of an arbitrary signal, this presumption often holds. However, this assumption may be invalid for some classification tasks, where other error metrics such as cross-entropy [38] or maximal class margin [39] may be more suited. Our paper focuses on regression-type problems.…”

Section: Overview Of Type-2 Fuzzy Logic Systemsmentioning

confidence: 99%

On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection

Zhou

Garibaldi

John

et al. 2009

IEEE Trans. Fuzzy Syst.

View full text Add to dashboard Cite

Abstract-Type-2 fuzzy systems are increasing in popularity and there are many examples of successful applications. While many techniques have been proposed for creating parsimonious type-1 fuzzy systems, there is a lack of such techniques for type-2 systems. The essential problem is to reduce the number of rules, whilst maintaining the system's approximation performance. In this paper, four novel indices for ranking the relative contribution of type-2 fuzzy rules are proposed, termed R-values, c-values, ω1-values and ω2-values. The R-values of type-2 fuzzy rules are obtained by applying a QR decomposition pivoting algorithm to the firing strength matrices of the trained fuzzy model. The cvalues rank rules based on the effects of rule consequents, whilst the ω1-values and ω2-values consider both the rule base structure (via firing strength matrices) and the output contribution of fuzzy rule consequents. Two procedures for utilising these indices in fuzzy rule selection (termed 'forward selection' and 'backward elimination') are described. Experiments are presented which demonstrate that, by using the proposed methodology, the most influential type-2 fuzzy rules can be effectively retained in order to construct parsimonious type-2 fuzzy models.

show abstract

“…Prior work has shown [8][9][10] that methods of calculating softer values for each training pattern based on the network's output vector improve generalization and reduce variance on classification problems over a corpus of benchmark learning problems. One of these, called lazy training or CB1, focuses on classification accuracy backpropagates an error signal through the network only when a pattern is misclassified.…”

Section: Motivation For Cb3mentioning

confidence: 99%

“…Classification-based (CB) error functions [9,10] are a relatively new method of training multi-layer perceptrons. The CB functions heuristically seek to directly minimize classification error by backpropagating network error only on misclassified patterns.…”

Section: Introductionmentioning

confidence: 99%

CB3: An Adaptive Error Function for Backpropagation Training

Rimer

Martinez

2006

Neural Process Lett

Self Cite

View full text Add to dashboard Cite

Effective backpropagation training of multi-layer perceptrons depends on the incorporation of an appropriate error or objective function. Classification-Based (CB) error functions are heuristic approaches that attempt to guide the network directly to correct pattern classification rather than using common error minimization heuristics, such as Sum-Squared Error (SSE) and Cross-Entropy (CE), which do not explicitly minimize classification error. This work presents CB3, a novel CB approach that learns the error function to be used while training. This is accomplished by learning pattern confidence margins during training, which are used to dynamically set output target values for each training pattern. On 11 applications, CB3 significantly outperforms previous CB error functions, and also reduces average test error over conventional error metrics using 0-1 targets without weight decay by 1.8%, and by 1.3% over metrics with weight decay. The CB3 also exhibits lower model variance and tighter mean confidence interval.

show abstract

Classification-based objective functions

Cited by 23 publications

References 17 publications

Fundamentals of Machine Learning

Fundamentals of Machine Learning

On Constructing Parsimonious Type-2 Fuzzy Logic Systems via Influential Rule Selection

CB3: An Adaptive Error Function for Backpropagation Training

Contact Info

Product

Resources

About