Abstract-A new methodology of extraction, optimization, and application of sets of logical rules is described. Neural networks are used for initial rule extraction, local, or global minimization procedures for optimization, and Gaussian uncertainties of measurements are assumed during application of logical rules. Algorithms for extraction of logical rules from data with real-valued features require determination of linguistic variables or membership functions. Context-dependent membership functions for crisp and fuzzy linguistic variables are introduced and methods of their determination described. Several neural and machine learning methods of logical rule extraction generating initial rules are described, based on constrained multilayer perceptron, networks with localized transfer functions or on separability criteria for determination of linguistic variables. A tradeoff between accuracy/simplicity is explored at the rule extraction stage and between rejection/error level at the optimization stage. Gaussian uncertainties of measurements are assumed during application of crisp logical rules, leading to "soft trapezoidal" membership functions and allowing to optimize the linguistic variables using gradient procedures. Numerous applications of this methodology to benchmark and real-life problems are reported and very simple crisp logical rules for many datasets provided.

Abstract

No abstract

No abstract

Abstract.A comprehensive methodology of extraction of optimal sets of logical rules using neural networks and global minimization procedures has been developed. Initial rules are extracted using density estimation neural networks with rectangular functions or multi-layered perceptron (MLP) networks trained with constrained backpropagation algorithm, transforming MLPs into simpler networks performing logical functions. A constructive algorithm called C-MLP2LN is proposed, in which rules of increasing specificity are generated consecutively by adding more nodes to the network. Neural rule extraction is followed by optimization of rules using global minimization techniques. Estimation of confidence of various sets of rules is discussed. The hybrid approach to rule extraction has been applied to a number of benchmark and real life problems with very good results. In many cases crisp logical rules are quite satisfactory, but sometimes fuzzy rules may be significantly more accurate.Keywords. Neural networks, fuzzy logic, data mining, extraction of logical rules Logical rules -introductionIn medicine the use of black box classifiers for diagnostic decision support is not acceptable. Adaptive systems M W , including the most popular neural network models, are useful classifiers that adjust internal parameters W performing vector mappings from the input to the output space. Although they may achieve high accuracy of classification the knowledge acquired by such systems is represented in a set of numerical parameters and architectures of networks in an incomprehensible way. In safety-critical applications such systems may suddenly break down Fuzziness and Soft Computing, Vol. 41 594 and thus are too dangerous to use. Even if other methods of classification -machine learning, pattern recognition or neural networks -may be easier to create and give good results logical rules should be preferred over other methods of classification provided that the set of rules is not too complex and classification accuracy is sufficiently high. Surprisingly, in many applications simple rules proved to be more accurate and were able to generalize better than various machine and neural learning algorithms. Springer (Studies inAlthough many statistical, pattern recognition and machine learning [1] methods of finding logical rules have been designed in the past automatic extraction of logical rules from data is still a hard problem. Recently neural network methods started to compete with better established machine learning and fuzzy logic rulebased methods. Unfortunately systematic comparison of neural, fuzzy and machine learning methods is still missing. Many neural rule extraction methods have recently been reviewed and compared experimentally [2], therefore we will not discuss them here. Neural methods focus on analysis of parameters (weights and biases) of trained networks, trying to achieve high fidelity of performance, i.e. similar results of classification by extracted logical rules and by the original networks. Non-standard form of...