Short-term cognitive networks, flexible reasoning and nonsynaptic learning

Nápoles, Gonzalo; Vanhoenshoven, Frank; Vanhoof, Koen

doi:10.1016/j.neunet.2019.03.012

Cited by 21 publications

(16 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aiming at conducting the simulations, we use the same datasets used in [3]. These datasets are traditional pattern classification problems adapted to the simulation problem where we need to predict the values of some variables from the values of others.…”

Section: Benchmark Problemsmentioning

confidence: 99%

See 1 more Smart Citation

On the Performance of the Nonsynaptic Backpropagation for Training Long-term Cognitive Networks

et al. 2021

View full text Add to dashboard Cite

Long-term Cognitive Networks (LTCNs) are recurrent neural networks for modeling and simulation. Such networks can be trained in a synaptic or nonsynaptic mode according to their goal. Nonsynaptic learning refers to adjusting the transfer function parameters while preserving the weights connecting the neurons. In that regard, the Nonsynaptic Backpropagation (NSBP) algorithm has proven successful in training LTCNbased models. Despite NSBP's success, a question worthy of investigation is whether the backpropagation process is necessary when training these recurrent neural networks. This paper investigates this issue and presents three nonsynaptic learning methods that modify the original algorithm. In addition, we perform a sensitivity analysis of both the NSBP's hyperparameters and the LTCNs' learnable parameters. The main conclusions of our study are i) the backward process attached to the NSBP algorithm is not necessary to train these recurrent neural systems, and ii) there is a nonsynaptic learnable parameter that does not contribute significantly to the LTCNs' performance.

show abstract

Section: Benchmark Problemsmentioning

confidence: 99%

“…To overcome the FCMs' drawbacks, Nápoles et al [3] introduced the Short-term Cognitive Networks and a learning principle referred to as nonsynaptic learning. Instead of adjusting the weight matrix, the nonsynaptic learning focuses on the transfer function parameters controlling neurons' excitation degree.…”

Section: Introductionmentioning

confidence: 99%

On the Performance of the Nonsynaptic Backpropagation for Training Long-term Cognitive Networks

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In this paper, we propose the Long Short-term Cognitive Networks (LSTCNs) to cope with the efficient and transparent forecasting of long univariate and multivariate time series. LSTCNs involve a sequential collection of Short-term Cognitive Network (STCN) blocks [34], each processing a specific time patch in the sequence. The STCN model allows for transparent reasoning since both weights and neurons map to specific features in the problem domain.…”

Section: Introductionmentioning

confidence: 99%

“…As a second contribution, we propose a deterministic learning algorithm to compute the tunable parameters of each STCN block in a deterministic fashion. The highly efficient algorithm replaces the non-synaptic learning method presented in [34]. As a final contribution, we present a feature influence score as a proxy to explain the reasoning process of our neural system.…”

Section: Introductionmentioning

confidence: 99%

Long Short-term Cognitive Networks

Nápoles¹,

Grau²,

Jastrzębska³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we present a recurrent neural system named Long Short-term Cognitive Networks (LSTCNs) as a generalisation of the Short-term Cognitive Network (STCN) model. Such a generalisation is motivated by the difficulty of forecasting very long time series in an efficient, greener fashion. The LSTCN model can be defined as a collection of STCN blocks, each processing a specific time patch of the (multivariate) time series being modelled. In this neural ensemble, each block passes information to the subsequent one in the form of a weight matrix referred to as the prior knowledge matrix. As a second contribution, we propose a deterministic learning algorithm to compute the learnable weights while preserving the prior knowledge resulting from previous learning processes. As a third contribution, we introduce a feature influence score as a proxy to explain the forecasting process in multivariate time series. The simulations using three case studies show that our neural system reports small forecasting errors while being up to thousands of times faster than state-of-the-art recurrent models.

show abstract

“…The proposed model relies on a brand new neural system termed Long Short-term Cognitive Network (LSTCN) [30] that allows for online learning. In this model, each iteration processes a data chunk using a Short-term Cognitive Network (STCN) block [31] that operates with the knowledge transferred from the previous block. This means that the model can easily be retrained without compromising what the networks has learned from previous data chunks.…”

Section: Introductionmentioning

confidence: 99%

Online learning of windmill time series using Long Short-term Cognitive Networks

Morales-Hernández¹,

Nápoles²,

Jastrzębska³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Forecasting windmill time series is often the basis of other processes such as anomaly detection, health monitoring or maintenance scheduling. The amount of data generated on windmill farms makes online learning the most viable strategy to follow. Such settings require retraining the model each time a new batch of data is available. However, update the model with the new information is often very expensive to perform using traditional Recurrent Neural Networks (RNNs). In this paper, we use Long Short-term Cognitive Networks (LSTCNs) to forecast windmill time series in online settings. These recently introduced neural systems consist of chained Short-term Cognitive Network blocks, each processing a temporal data chunk. The learning algorithms of these blocks is based on a very fast, deterministic learning rule that makes LSTCNs suitable for online learning tasks. The numerical simulations using a case study with four windmills showed that our approach reported the lowest forecasting errors with respect to a simple RNN, a Long Short-term Memory, a Gated Recurrent Unit and a Hidden Markov Model. What is perhaps more important is that the LSTCN approach is significantly faster than these state-of-the-art models.

show abstract

Short-term cognitive networks, flexible reasoning and nonsynaptic learning

Cited by 21 publications

References 33 publications

On the Performance of the Nonsynaptic Backpropagation for Training Long-term Cognitive Networks

On the Performance of the Nonsynaptic Backpropagation for Training Long-term Cognitive Networks

Long Short-term Cognitive Networks

Online learning of windmill time series using Long Short-term Cognitive Networks

Contact Info

Product

Resources

About