Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]

Arel, Itamar; Rose, Derek; Karnowski, Thomas P.

doi:10.1109/mci.2010.938364

Cited by 1,049 publications

(406 citation statements)

References 36 publications

Supporting

Mentioning

401

Contrasting

Unclassified

Order By: Relevance

“…5.6.4) was formulated as a combination of convex optimization problems (Lee et al, 2007a). Recent surveys of stacked RBM and AE methods focus on post-2006 developments (Bengio, 2009;Arel et al, 2010). Unsupervised DBNs and AE stacks are conceptually similar to, but in a certain sense less general than, the unsupervised RNN stack-based History Compressor of 1991 (Sec.…”

Section: /7: Ul For Deep Belief Network / Ae Stacks Fine-tuned Bmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

Section: /7: Ul For Deep Belief Network / Ae Stacks Fine-tuned Bmentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

show abstract

“…The commonly used models for dealing with temporal information based on Hidden Markov Models (HMM) [88] and traditional artificial neural networks (ANN) [57] have limited capacity to achieve the integration of complex and long temporal spatial/spectral components because they usually either ignore the temporal dimension or over-simplify its representation. A new trend in machine learning is currently emerging and is known as deep machine learning [9,[2][3][4]112]. Most of the proposed models still learn SSTD by entering single time point frames rather than learning whole SSTD patterns.…”

Section: Evolving Spiking Neural Network and Neurogenetic Systems Fomentioning

confidence: 99%

Evolving Spiking Neural Networks and Neurogenetic Systems for Spatio- and Spectro-Temporal Data Modelling and Pattern Recognition

Kasabov

2012

Advances in Computational Intelligence

View full text Add to dashboard Cite

“…Recent advances in computer vision and machine learning have suggested that spatial feature representation can be learnt hierarchically at multiple levels through deep learning algorithms [15]. These deep learning approaches learn the spatial contexts at higher levels through the models themselves to achieve enhanced generalization capabilities.…”

Section: Introductionmentioning

confidence: 99%