Comparison of Echo State Networks with Simple Recurrent Networks and Variable-Length Markov Models on Symbolic Sequences

Čerňanský, Michal; Tiňo, Peter

doi:10.1007/978-3-540-74690-4_63

Cited by 7 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, training of LSTM as an RNN-based model is reported to be inherently difficult (Lukoševičius, 2012). As a consequent, in this paper, we have proposed ESN architecture as a new powerful approach in RNN research, where, instead of difficult learning process, it based on the property of untrained randomly initialized RNN (Čerňanský and Tiňo, 2007). We shall see in section 5 that the ESN can significantly reduce the training time and achieve results comparable to the LSTM.…”

Section: Literature Reviewmentioning

confidence: 98%

Efficient Kinect Sensor-based Kurdish Sign Language Recognition Using Echo System Network

Mirza

Al-Talabani

2021

ARO

View full text Add to dashboard Cite

Sign language assists in building communication and bridging gaps in understanding. Automatic sign language recognition (ASLR) is a field that has recently been studied for various sign languages. However, Kurdish sign language (KuSL) is relatively new and therefore researches and designed datasets on it are limited. This paper has proposed a model to translate KuSL into text and has designed a dataset using Kinect V2 sensor. The computation complexity of feature extraction and classification steps, which are serious problems for ASLR, has been investigated in this paper. The paper proposed a feature engineering approach on the skeleton position alone to provide a better representation of the features and avoid the use of all of the image information. In addition, the paper proposed model makes use of recurrent neural networks (RNNs)-based models. Training RNNs is inherently difficult, and consequently, motivates to investigate alternatives. Besides the trainable long short-term memory (LSTM), this study has proposed the untrained low complexity echo system network (ESN) classifier. The accuracy of both LSTM and ESN indicates they can outperform those in state-of-the-art studies. In addition, ESN which has not been proposed thus far for ASLT exhibits comparable accuracy to the LSTM with a significantly lower training time.

show abstract

Section: Literature Reviewmentioning

confidence: 98%

Efficient Kinect Sensor-based Kurdish Sign Language Recognition Using Echo System Network

Mirza

Al-Talabani

2021

ARO

View full text Add to dashboard Cite

show abstract

“…Methods used for the classification span from distance measures (e.g., Dynamic Time Warping) [11,12] and statistical models (e.g., Hidden Markov Models) [13,14], to artificial neural architectures (e.g., Recurrent Neural Networks) [15][16][17][18][19] and hybrid solutions [20]. These methods vary in complexity and adaptability, with Recurrent Neural Networks being one of the most promising direction in the field [21]. Adaptation of RNNs though, is known to have high computational complexity.…”

Section: Introductionmentioning

confidence: 99%

Effective Behavioural Dynamic Coupling through Echo State Networks

Melidis

Marocco

2019

Applied Sciences

View full text Add to dashboard Cite

This work presents a novel approach and paradigm for the coupling of human and robot dynamics with respect to control. We present an adaptive system based on Reservoir Computing and Recurrent Neural Networks able to couple control signals and robotic behaviours. A supervised method is utilised for the training of the network together with an unsupervised method for the adaptation of the reservoir. The proposed method is tested and analysed using a public dataset, a set of dynamic gestures and a group of users under a scenario of robot navigation. First, the architecture is benchmarked and placed among the state of the art. Second, based on our dataset we provide an analysis for key properties of the architecture. We test and provide analysis on the variability of the lengths of the trained patterns, propagation of geometrical properties of the input signal, handling of transitions by the architecture and recognition of partial input signals. Based on the user testing scenarios, we test how the architecture responds to real scenarios and users. In conclusion, the synergistic approach that we follow shows a way forward towards human in-the-loop systems and the evidence provided establish its competitiveness with available methods, while the key properties analysed the merits of the approach to the commonly used ones. Finally, reflective remarks on the applicability and usage in other fields are discussed.

show abstract

“…These results were repeated by Tong et al [4] who compared SRNs with a recent popular model, echo-state networks (ESN), introduced by Jaeger [5]. It has been shown that ESN performance is similar to common statistical methods (variable-length Markov models) while a well-trained SRN can demonstrate superior prediction abilities [6]. 1 What all these modeling approaches share is their focus on natural language processing as an independent domain.…”

Section: Introductionmentioning

confidence: 99%

Modeling Utterance-Driven Visual Attention During Situated Comprehension

Švantner

Farkaš

Crocker

2012

NNW

View full text Add to dashboard Cite

Genetic algorithms (GAs) are stochastic methods that are widely used in search and optimization. The breeding process is the main driving mechanism for GAs that leads the way to find the global optimum. And the initial phase of the breeding process starts with parent selection. The selection utilized in a GA is effective on the convergence speed of the algorithm. A GA can use different selection mechanisms for choosing parents from the population and in many applications the process generally depends on the fitness values of the individuals. Artificial neural networks (ANNs) are used to decide the appropriate parents by the new hybrid algorithm proposed in this study. And the use of neural networks aims to produce better offspring during the GA search. The neural network utilized in this algorithm tries to learn the structural patterns and correlations that enable two parents to produce high-fit offspring. In the breeding process, the first parent is selected based on the fitness value as usual. Then it is the neural network that decides the appropriate mate for the first parent chosen. Hence, the selection mechanism is not solely dependent on the fitness values in this study. The algorithm is tested with seven benchmark functions. It is observed from results of these tests that the new selection method leads genetic algorithm to converge faster.

show abstract

Comparison of Echo State Networks with Simple Recurrent Networks and Variable-Length Markov Models on Symbolic Sequences

Cited by 7 publications

References 16 publications

Efficient Kinect Sensor-based Kurdish Sign Language Recognition Using Echo System Network

Efficient Kinect Sensor-based Kurdish Sign Language Recognition Using Echo System Network

Effective Behavioural Dynamic Coupling through Echo State Networks

Modeling Utterance-Driven Visual Attention During Situated Comprehension

Contact Info

Product

Resources

About