Computational intelligence methods for rule-based data understanding

Studies in Computational Intelligence

Maszczyk

Grochowski

2011

Meta-learning has many aspects, but its final goal is to discover in an automatic way many interesting models for a given data. Our early attempts in this area involved heterogeneous learning systems combined with a complexity-guided search for optimal models, performed within the framework of (dis)similarity based methods to discover "knowledge granules". This approach, inspired by neurocognitive mechanisms of information processing in the brain, is generalized here to learning based on parallel chains of transformations that extract useful information granules and use it as additional features. Various types of transformations that generate hidden features are analyzed and methods to generate them are discussed. They include restricted random projections, optimization of these features using projection pursuit methods, similarity-based and general kernel-based features, conditionally defined features, features derived from partial successes of various learning algorithms, and using the whole learning models as new features. In the enhanced feature space the goal of learning is to create image of the input data that can be directly handled by relatively simple decision processes. The focus is on hierarchical methods for generation of information, starting from new support features that are discovered by different types of data models created on similar tasks and successively building more complex features on the enhanced feature spaces. Resulting algorithms facilitate deep learning, and also enable understanding of structures present in the data by visualization of the results of data transformations and by creating logical, fuzzy and prototype-based rules based on new features. Relations to various machine-learning approaches, comparison of results, and neurocognitive inspirations for meta-learning are discussed.

Section: Extracting Features For Meta-learningmentioning

confidence: 99%

Section: Binary Featuresmentioning

confidence: 99%

Section: Transfer Of Knowledgementioning

confidence: 99%

See 1 more Smart Citation

Optimal Support Features for Meta-Learning

Studies in Computational Intelligence

Maszczyk

Grochowski

2011

“…Decisions of neural networks (or other models) that learn from data frequently cannot be justified in terms of logical rules. In some cases logical rules that have similar or even higher predictive power may be extracted from trained neural networks [11]. In other cases judgments based on overall similarity provide better decisions.…”

Section: Intuitionmentioning

confidence: 99%

Intuition, Insight, Imagination and Creativity

IEEE Comput. Intell. Mag.

2007

Abstract-Can computers have intuition, insights and be creative? Neurocognitive models inspired by the putative processes in the brain show that these mysterious features are a consequence of information processing in complex networks. Intuition is manifested in categorization based on evaluation of similarity, when decision borders are too complex to be reduced to logical rules. It is also manifested in heuristic reasoning based on partial observations, where network activity selects only those path that may lead to solution, excluding all bad moves. Insight results from reasoning at the higher, non-verbal level of abstraction that comes from involvement of the right hemisphere networks forming large "linguistic receptive fields". Three factors are essential for creativity in invention of novel words: knowledge of word morphology captured in network connections, imagination constrained by this knowledge, and filtering of results that selects most interesting novel words. These principles have been implemented using a simple correlation-based algorithm for autoassociative memory. Results are surprisingly similar to those created by humans.Keywords-Creativity, intuition, insight, brain, language processing, higher cognitive functions, neural modeling.One of the objections against computational intelligence considered by Alan Turing in his famous article "Computing machinery and intelligence" [1] recalls Lady Lovelace's objection (written in her memoirs in 1842) that a machine can "never do anything really new", and in particular the Analytical Engine of Babbage (an early idea for universal computer) "has no pretensions to originate anything. It can do whatever we know how to order it to perform". Turing's response can be summarized as: "the evidence available to Lady Lovelace did not encourage her to believe" that machines could be creative, although "It is quite possible that the machines in question had in a sense got this property" because "suppose that some discrete-state machine has the property. … universal digital computer … could by suitable programming be made to mimic the machine in question". It is difficult to ascertain that something is really new, and Turing admits that "Machines take me by surprise with great frequency".The last section of Turing's article is devoted to learning machines as our best hope to realize computational intelligence and creativity. After proposing (albeit in a very vague terms) "the child machine" in the final paragraph of the paper the author writes: "We may hope that machines will eventually compete with men in all purely intellectual fields. But which are the best ones to start with? Even this is a difficult decision. Many people think that a very abstract activity, like the playing of chess, would be best." This has indeed proved to be true and before the turn of the century -as Turing predicted -computers exceeded human level competence in chess. However, the connection between memory capacity and speed of calculations in chess is quite obvious, therefore the famous B...

“…Furthermore, simplest models of the data should be thought to avoid overfitting and ensure good generalization. Often quite complex data may be described using a few simple rules [4], therefore using an appropriate constructive strategy should create network with small number of neurons and clear interpretation.…”

Section: Introductionmentioning

confidence: 99%

A Comparison of Methods for Learning of Highly Non-separable Problems

Grochowski

Artificial Intelligence and Soft Computing – ICAISC 2008

Abstract. Learning in cases that are almost linearly separable is easy, but for highly non-separable problems all standard machine learning methods fail. Many strategies to build adaptive systems are based on the "divide-and-conquer" principle. Constructive neural network architectures with novel training methods allow to overcome some drawbacks of standard backpropagation MLP networks. They are able to handle complex multidimensional problems in reasonable time, creating models with small number of neurons. In this paper a comparison of our new constructive c3sep algorithm based on k-separability idea with several sequential constructive learning methods is reported. Tests have been performed on parity function, 3 artificial Monks problems, and a few benchmark problems. Simple and accurate solutions have been discovered using c3sep algorithm even in highly non-separable cases.