Deep Learning

Abstract

Any fool can know. The point is to understand. -Albert EinsteinArtificial neural networks (ANNs) have had a history riddled with highs and lows since their inception. At a nodal level, ANNs started with highly simplified neural models, such as McCulloch-Pitts neurons (McCulloch and Pitts 1943), and then evolved into Rosenblatt's perceptrons (Rosenblatt 1957) and a variety of more complex and sophisticated computational units. From single-and multilayer networks, to self-recurrent Hopfield networks (Tank and Hopfield 1986), to self-organizing maps (also called Kohonen networks) (Kohonen 1982), adaptive resonance theory and time delay neural networks among other recommendations, ANNs have witnessed many structural iterations. These generations carried incremental enhancements that promised to address predecessors' limitations and achieve higher levels of intelligence. Nonetheless, the compounded effect of these "intelligent" networks has not been able to capture the true human intelligence (Guerriere and Detsky 1991; Becker and Hinton 1992). Thus, Deep learning is on the rise in the machine learning community, because the traditional shallow learning architectures have proved unfit for the more challenging tasks of machine learning and strong artificial intelligence (AI). The surge in and wide availability of increased computing power (Misra and Saha 2010), coupled with the creation of efficient training algorithms and advances in neuroscience, have enabled the implementation, hitherto impossible, of deep learning principles. These developments have led to the formation of deep architecture algorithms that look in to cognitive neuroscience to suggest biologically inspired learning solutions. This chapter presents the concepts of spiking neural networks (SNNs) and hierarchical temporal memory (HTM), whose associated techniques are the least mature of the techniques covered in this book. Overview of Hierarchical Temporal MemoryHTM aims at replicating the functional and structural properties of the neocortex. HTM incorporates a number of insights from Hawkins's book On Intelligence (2007), which postulates that the key to intelligence is the ability to predict. Its framework was designed as a biomimetic model of the neocortex that seeks to replicate the brain's structural and algorithmic properties, albeit in a simplified, functionally oriented manner. HTM is therefore organized hierarchically, as depicted generically in Figure 9-1. All levels of hierarchy and their subcomponents perform a common computational algorithm.

Machine learning for algal biofuels: a critical review and perspective for the future

“…Then, the model development step can be implemented by starting with selecting the algorithms to be used (highly dependent on the purpose of the research and characteristic of the data); regression is the most common task performed through algorithms like simple regression, artificial neural networks (and deep learning networks), support vector machines, regression trees, and gradient boosting trees. 71 Classification is another frequently implemented task (usually for categorical variables) through the algorithms such as decision trees (DTs), support vector machines (SVMs) and logistic regression (LogR). 72 The common practice in model building is that the dataset is divided into two parts, namely training (model building) and testing (verification of model performance); the training is performed using the first set together with a validation procedure (like k-fold cross-validation) to select the best model ( i.e.…”

Systematic process for crop insurance development: area-yield rice insurance with machine learning technology implementation in Thailand

“…Using a R -square figure shows that polynomial regression has a noticeably better curve fitting than one of linear regression (Maulud and Abdulazeez, 2020). Support vector regression, generalized from support vector machine concepts, is an effective tool for continuous value prediction (Awad and Khanna, 2015). It has advantages of high dimensional of features (Drucker et al.…”

Stable sparse encoding for predictive processing

“…Interest in HTM has steadily grown, with various researchers now turning their attention to the model. The history of HTM development and reviews of some of the HTM research are discussed in the article by Awad and Khanna [17]. Currently, much of the work is still focussed on the SP [18], [19], with several mathematical formulations of the SP and its computational properties recently being published [20], [21], [22].…”