Combining symbolic and neural learning

Shavlik, Jude W.

doi:10.1007/bf00993982

Cited by 62 publications

(50 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The NN's depth reflects the longest chain of reasoning in the original set of logical rules. An extension of this approach (Maclin and Shavlik, 1993;Shavlik, 1994) initializes an RNN by domain knowledge expressed as a Finite State Automaton (FSA). BP-based fine-tuning has become important for later DL systems pre-trained by UL, e.g., Sec.…”

Section: Ideas For Dealing With Long Time Lags and Deep Capsmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

Section: Ideas For Dealing With Long Time Lags and Deep Capsmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

show abstract

“…The extraction of symbolic knowledge from trained artificial neural networks permits the exchange of information between connectionist and symbolic knowledge representations and has been of great interest to understand what the artificial neural network actually does [Sha94]. Additionally, a significant decrease in training time can be obtained by training networks with initial knowledge [OG96].…”

Section: Symbolic-connectionist Hybrid Systemsmentioning

confidence: 99%

Biologically plausible connectionist prediction of natural language thematic relations

Rosa

2010

The 2010 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Abstract:In Natural Language Processing (NLP) symbolic systems, several linguistic phenomena, for instance, the thematic role relationships between sentence constituents, such as agent, patient, and location, can be accounted for by the employment of a rule-based grammar. Another approach to NLP concerns the use of the connectionist model, which has the benefits of learning, generalization and fault tolerance, among others. A third option merges the two previous approaches into a hybrid one: a symbolic thematic theory is used to supply the connectionist network with initial knowledge. Inspired on neuroscience, it is proposed a symbolic-connectionist hybrid system called BioθPred (Biologically plausible thematic (θ) symbolic-connectionist Predictor ), designed to reveal the thematic grid assigned to a sentence. Its connectionist architecture comprises, as input, a featural representation of the words (based on the verb/noun WordNet classification and on the classical semantic microfeature representation), and, as output, the thematic grid assigned to the sentence. BioθPred is designed to "predict" thematic (semantic) roles assigned to words in a sentence context, employing biologically inspired training algorithm and architecture, and adopting a psycholinguistic view of thematic theory.

show abstract

“…[8], [14] and [15] provide motivation in the direction of knowledge based architectures. With a lot of work undertaken in the area of text classification, hybrid classifiers [16]- [22] and boosting enabled artificial network systems [23]- [27] have also been investigated to improve classification performance.…”

Section: A Hybrid Neural/connectionist Architecturesmentioning

confidence: 99%

Boosted Hybrid Recurrent Neural Classifier for Text Document Classification on the Reuters News Text Corpus

Buabin¹

2012

IJMLC

View full text Add to dashboard Cite

Abstract-The objective is multi-classed news text classification using hybrid neural techniques on the modapte version of the Reuters news text corpus. In particular, a neuroscience based hybrid neural classifier fully integrated with a novel boosting algorithm is examined for its potential in text document classification in a non-stationary environment. The novel boosting algorithm termed NeuroBoost is an Adaboost-like algorithm that computes and integrates boosted weights into neural network weights, using back-propagation approach. The main contribution of this paper is the provision of an obvious scientific basis for integrating boosted weights into hybrid neural network weights. Results attained in this experiment show impressive performance by the hybrid neural classifier even with minimal number of neurons in constituting structures. A minimal but appreciable increase is observed in performance if an appreciable number of neurons are added.Index Terms-Hybrid neural classifier, intelligent agent, natural language processing, recurrent neural network, text classification.

show abstract

Combining symbolic and neural learning

Cited by 62 publications

References 36 publications

Deep learning in neural networks: An overview

Deep learning in neural networks: An overview

Biologically plausible connectionist prediction of natural language thematic relations

Boosted Hybrid Recurrent Neural Classifier for Text Document Classification on the Reuters News Text Corpus

Contact Info

Product

Resources

About