Going Beyond Linear Transformers with Recurrent Fast Weight Programmers

Irie, Kazuki; Schlag, Imanol; Csordás, Róbert; Schmidhuber, Jürgen

doi:10.48550/arxiv.2106.06295

Cited by 3 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Parisotto et al (2020) address the problem of using transformers in RL and showed that adding gating layers on top of the transformers layers can stabilize training. Subsequent works addressed the increased computational load of using a transformer for an agent's policy (Irie et al, 2021;Parisotto & Salakhutdinov, 2021). Chen et al (2021); Janner et al (2021) take a different approach by modeling the RL problem as a sequence modeling problem and use a transformer to predict actions without additional networks for an actor or critic.…”

Section: Related Workmentioning

confidence: 99%

TransDreamer: Reinforcement Learning with Transformer World Models

Chen¹,

Yoon²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

The Dreamer agent provides various benefits of Model-Based Reinforcement Learning (MBRL) such as sample efficiency, reusable knowledge, and safe planning. However, its world model and policy networks inherit the limitations of recurrent neural networks and thus an important question is how an MBRL framework can benefit from the recent advances of transformers and what the challenges are in doing so. In this paper, we propose a transformer-based MBRL agent, called TransDreamer. We first introduce the Transformer State-Space Model, a world model that leverages a transformer for dynamics predictions. We then share this world model with a transformer-based policy network and obtain stability in training a transformer-based RL agent. In experiments, we apply the proposed model to 2D visual RL and 3D first-person visual RL tasks both requiring long-range memory access for memory-based reasoning. We show that the proposed model outperforms Dreamer in these complex tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

TransDreamer: Reinforcement Learning with Transformer World Models

Chen¹,

Yoon²,

Wu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Liu et al propose a solution to the vanishing gradient problem in [51]. However, both networks are very complex and need a long time for training and to become effective [52].…”

Section: The Second Strategy: Distilbert Language Model (Transformers...mentioning

confidence: 99%

AI-Crime Hunter: An AI Mixture of Experts for Crime Discovery on Twitter

Shoeibi

Hernández

et al. 2021

Electronics

View full text Add to dashboard Cite

Maintaining a healthy cyber society is a great challenge due to the users’ freedom of expression and behavior. This can be solved by monitoring and analyzing the users’ behavior and taking proper actions. This research aims to present a platform that monitors the public content on Twitter by extracting tweet data. After maintaining the data, the users’ interactions are analyzed using graph analysis methods. Then, the users’ behavioral patterns are analyzed by applying metadata analysis, in which the timeline of each profile is obtained; also, the time-series behavioral features of users are investigated. Then, in the abnormal behavior detection and filtering component, the interesting profiles are selected for further examinations. Finally, in the contextual analysis component, the contents are analyzed using natural language processing techniques; a binary text classification model (SVM (Support Vector Machine) + TF-IDF (Term Frequency—Inverse Document Frequency) with 88.89% accuracy) is used to detect if a tweet is related to crime or not. Then, a sentiment analysis method is applied to the crime-related tweets to perform aspect-based sentiment analysis (DistilBERT + FFNN (Feed-Forward Neural Network) with 80% accuracy), because sharing positive opinions about a crime-related topic can threaten society. This platform aims to provide the end-user (the police) with suggestions to control hate speech or terrorist propaganda.

show abstract

“…Since then the applications of DNNs have been increasing dramatically as the advanced graphic processing units (GPUs) and a significant boost in computing power. From 2009 to 2012, Jurgen Schmidhuber of IDSIA, a Swiss AI laboratory, developed feedforward neural network (FNN) [9], [10]. Hinton et al won the ImagNet 2012 [11], outperformed the second-place competitors in image classifying precision, therefore, leading to the current deep learning boom.…”

Section: Introductionmentioning

confidence: 99%

Fuzzy C-Means Clustering Based Deep Patch Learning With Improved Interpretability for Classification Problems

et al. 2022

View full text Add to dashboard Cite

Grid partitioning for input space results in the exponential rise in the number of rules in adaptive network-based fuzzy inference system (ANFIS) and patch learning (PL) as the number of features increases, thus resulting in the huge computational load and deteriorating its interpretability. An improved PL (iPL) is put forward for the training of each sub-fuzzy system to overcome the rule-explosion problem. In the iPL, input partitioning is done using fuzzy c-means (FCM) clustering to avoid the heavy computational complexity arising due to the large number of rules generated from high dimensionality. In this paper, two novel classifiers, called FCM clustering based deep patch learning with improved high-level interpretability for classification problems, are presented, named as HI-FCMDPL-CP1 and HI-FCMDPL-CP2. The proposed classifiers have two characteristics: One is a stacked deep structure of component iPL fuzzy classifiers for high accuracy, and the other is the use of maximal information coefficient (MIC) and the maximum misclassification threshold (MMT) to optimize the deep structures. High interpretability is achieved at each layer by using the FCM clustering, concise structure and large input dimensionality. The MMT, random input (RI) and parameter sharing (PS) are integrated to improve their classification accuracy without losing their interpretability. Experiments on several real-word datasets demonstrated that MIC, RI and PS in HI-FCMDPL-CP1 and HI-FCMDPL-CP2 are effective individually, and integrating them all three can further improve the classification performance. A more concise deep fuzzy system is obtained with the number of features and fuzzy rules reduced simultaneously. Furthermore, MIC, RI and PS are used to determine the advantages and disadvantages of using serial versus parallel structures to avoid subjective selection of these two categories.INDEX TERMS Fuzzy c-means (FCM) clustering, maximal information coefficient (MIC), random input (RI), deep patch learning classifier, interpretability.

show abstract

Going Beyond Linear Transformers with Recurrent Fast Weight Programmers

Cited by 3 publications

References 42 publications

TransDreamer: Reinforcement Learning with Transformer World Models

TransDreamer: Reinforcement Learning with Transformer World Models

AI-Crime Hunter: An AI Mixture of Experts for Crime Discovery on Twitter

Fuzzy C-Means Clustering Based Deep Patch Learning With Improved Interpretability for Classification Problems

Contact Info

Product

Resources

About