Continuous Recurrent Neural Networks Based on Function Satlins

Huang, Yué; Yu, Jiali; Leng, Jinsong; Liu, Bisen; Zhang, Yi

doi:10.1007/s11063-021-10682-9

Cited by 3 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work in computer vision takes inspiration from PC theory to build models for accurate (Han et al, 2018) and robust (Huang et al, 2020) image classification. PredNet (Lotter et al, 2017) proposes a network capable of predicting future frames in a video sequence by making local predictions at each level using top-down connections.…”

Section: Predictive Coding and Deep Learningmentioning

confidence: 99%

Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations

Araujo¹,

Villa²,

Mendoza³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Current language models are usually trained using a self-supervised scheme, where the main focus is learning representations at the word or sentence level. However, there has been limited progress in generating useful discourse-level representations. In this work, we propose to use ideas from predictive coding theory to augment BERT-style language models with a mechanism that allows them to learn suitable discourse-level representations. As a result, our proposed approach is able to predict future sentences using explicit top-down connections that operate at the intermediate layers of the network. By experimenting with benchmarks designed to evaluate discourse-related knowledge using pre-trained sentence representations, we demonstrate that our approach improves performance in 6 out of 11 tasks by excelling in discourse relationship detection.

show abstract

Section: Predictive Coding and Deep Learningmentioning

confidence: 99%

Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations

Araujo¹,

Villa²,

Mendoza³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…Given a speech dataset of the clean-noisy pairs, the neural networks can learn to transform the noisy magnitude spectra to their clean counterparts (mapping based) [ 12 – 14 ] or estimate the time-frequency masks (masking-based) such as ideal binary mask (IBM) [ 15 , 16 ], ideal ratio mask (IRM) [ 17 , 18 ], and spectral magnitude mask (SMM) [ 19 ]. Fully connected networks (FCN) [ 19 ], feedforward neural networks (FDNN) with Kalman filtering [ 20 ], recurrent neural networks (RNN) [ 21 – 23 ], and convolutional neural networks (CNN) [ 24 , 25 ] are important deep learning approaches in SE.…”

Section: Introductionmentioning

confidence: 99%

Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network

Peracha,

Khattak,

Salem

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Speech enhancement (SE) reduces background noise signals in target speech and is applied at the front end in various real-world applications, including robust ASRs and real-time processing in mobile phone communications. SE systems are commonly integrated into mobile phones to increase quality and intelligibility. As a result, a low-latency system is required to operate in real-world applications. On the other hand, these systems need efficient optimization. This research focuses on the single-microphone SE operating in real-time systems with better optimization. We propose a causal data-driven model that uses attention encoder-decoder long short-term memory (LSTM) to estimate the time-frequency mask from a noisy speech in order to make a clean speech for real-time applications that need low-latency causal processing. The encoder-decoder LSTM and a causal attention mechanism are used in the proposed model. Furthermore, a dynamical-weighted (DW) loss function is proposed to improve model learning by varying the weight loss values. Experiments demonstrated that the proposed model consistently improves voice quality, intelligibility, and noise suppression. In the causal processing mode, the LSTM-based estimated suppression time-frequency mask outperforms the baseline model for unseen noise types. The proposed SE improved the STOI by 2.64% (baseline LSTM-IRM), 6.6% (LSTM-KF), 4.18% (DeepXi-KF), and 3.58% (DeepResGRU-KF). In addition, we examine word error rates (WERs) using Google’s Automatic Speech Recognition (ASR). The ASR results show that error rates decreased from 46.33% (noisy signals) to 13.11% (proposed) 15.73% (LSTM), and 14.97% (LSTM-KF).

show abstract

Method of Location and Capacity Determination of Intelligent Charging Pile Based on Recurrent Neural Network

2022

WEVJ

View full text Add to dashboard Cite

With the popularity of new energy vehicles, a large number of cities began to focus on the installation of electric vehicle charging piles. However, the existing intelligent charging piles have faced problems such as short supply, unreasonable distribution areas, and insufficient power supply. In response to these problems, this research proposes a recurrent neural network algorithm with an integrated firefly algorithm. Based on these two algorithms, a charging pile location and capacity model was established, and users' travel habits were analyzed according to the model. In the simulation experiment, the PR curve analysis of the algorithm was carried out first. The analysis results showed that the AP value of the recurrent neural network algorithm combined with the firefly algorithm was increased from 0.9324 to 0.9972. In addition, it had higher accuracy and stability than before, which also verified the feasibility of the algorithm. Finally, through the model, the user's travel habits were analyzed in detail. From the perspective of total demand, the charging demand of commercial centers was the highest, with a peak of about 537 kw, followed by 501 kw in office areas and then about 379 kw in parks. The kw charging demand in other areas was below 200 kw. The above results show that the recursive neural network can effectively determine the location and capacity of the charging pile, which is of great value to the development of transportation and new energy.

show abstract

Continuous Recurrent Neural Networks Based on Function Satlins

Cited by 3 publications

References 31 publications

Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations

Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations

Causal speech enhancement using dynamical-weighted loss and attention encoder-decoder recurrent neural network

Method of Location and Capacity Determination of Intelligent Charging Pile Based on Recurrent Neural Network

Contact Info

Product

Resources

About