Economic LSTM Approach for Recurrent Neural Networks

Khalil, Kasem; Eldash, Omar; Kumar, Ashok; Bayoumi, Magdy

doi:10.1109/tcsii.2019.2924663

Cited by 59 publications

(15 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The GAPG method is composed of two networks. The irst is a recurrent neural network (RNN) that takes long short-term memory (LSTM) [39] as the concrete implementation. This network acts as a generator G that receives a set of veriication properties as its initial input; then, it tries to generate veriication properties that are suiciently similar to the input data.…”

Section: Cgan the Original Gan Can Perform Only Generation Based On R...mentioning

confidence: 99%

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

Gao

Dai

Miao

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Formal methods have been widely used to support software testing to guarantee correctness and reliability. For example, model checking technology attempts to ensure that the verification property of a specific formal model is satisfactory for discovering bugs or abnormal behavior from the perspective of temporal logic. However, because automatic approaches are lacking, a software developer/tester must manually specify verification properties. A generative adversarial network (GAN) learns features from input training data and outputs new data with similar or coincident features. GANs have been successfully used in the image processing and text processing fields and achieved interesting and automatic results. Inspired by the power of GANs, in this paper, we propose a GAN-based automatic property generation (GAPG) approach to generate verification properties supporting model checking. First, the verification properties in the form of computational tree logic (CTL) are encoded and used as input to the GAN. Second, we introduce regular expressions as grammar rules to check the correctness of the generated properties. These rules work to detect and filter meaningless properties that occur because the GAN learning process is uncontrollable and may generate unsuitable properties for real applications. Third, the learning network is further trained by using labeled information associated with the input properties. These are intended to guide the training process to generate additional new properties, particularly those that map to corresponding formal models. Finally, a series of comprehensive experiments demonstrate that the proposed GAPG method can obtain new verification properties from two aspects: 1) using only CTL formulas and 2) using CTL formulas combined with Kripke structures.

show abstract

Section: Cgan the Original Gan Can Perform Only Generation Based On R...mentioning

confidence: 99%

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

Gao

Dai

Miao

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

“…Similar to GRU, using a more compact recursion formula to reduce the number of gates can simplify the networks and have comparable performance with LSTM. For example, Khalil et al [23] combine the forget gate G fg , input gate G in , and output gate G ot into one gate, and let the combined gate and the tanh gate to include the internal state α t−1 as the peephole connections. The proposed network is, thus, requiring fewer hardware units to perform its functionality.…”

Section: Modified Peephole Lstmmentioning

confidence: 99%

Explicit Duration Recurrent Networks

Shu

2022

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Recurrent neural networks (RNNs) can be used to operate over sequences of vectors and have been successfully applied to a variety of problems. However, it is hard to use RNNs to model the variable dwell time of the hidden state underlying an input sequence. In this article, we interpret the typical RNNs, including original RNN, standard long short-term memory (LSTM), peephole LSTM, projected LSTM, and gated recurrent unit (GRU), using a slightly extended hidden Markov model (HMM). Based on this interpretation, we are motivated to propose a novel RNN, called explicit duration recurrent network (EDRN), analog to a hidden semi-Markov model (HSMM). It has a better performance than conventional LSTMs and can explicitly model any duration distribution function of the hidden state. The model parameters become interpretable and can be used to infer many other quantities that the conventional RNNs cannot obtain. Therefore, EDRN is expected to extend and enrich the applications of RNNs. The interpretation also suggests that the conventional RNNs, including LSTM and GRU, can be made small modifications to improve their performance without increasing the parameters of the networks.

show abstract

“…The RNN has been widely used to handle the longitudinal variables, LSTM is one type of RNN (24,25). It can effectively process a large amount of sequential data.…”

Section: Recurrent Neural Network-long Short-term Memorymentioning

confidence: 99%

Application of Machine Learning to Predict Acute Kidney Disease in Patients With Sepsis Associated Acute Kidney Injury

Lin

Duan

2021

Front. Med.

View full text Add to dashboard Cite

Background: Sepsis-associated acute kidney injury (AKI) is frequent in patients admitted to intensive care units (ICU) and may contribute to adverse short-term and long-term outcomes. Acute kidney disease (AKD) reflects the adverse events developing after AKI. We aimed to develop and validate machine learning models to predict the occurrence of AKD in patients with sepsis-associated AKI.Methods: Using clinical data from patients with sepsis in the ICU at Beijing Friendship Hospital (BFH), we studied whether the following three machine learning models could predict the occurrence of AKD using demographic, laboratory, and other related variables: Recurrent Neural Network-Long Short-Term Memory (RNN-LSTM), decision trees, and logistic regression. In addition, we externally validated the results in the Medical Information Mart for Intensive Care III (MIMIC III) database. The outcome was the diagnosis of AKD when defined as AKI prolonged for 7–90 days according to Acute Disease Quality Initiative-16.Results: In this study, 209 patients from BFH were included, with 55.5% of them diagnosed as having AKD. Furthermore, 509 patients were included from the MIMIC III database, of which 46.4% were diagnosed as having AKD. Applying machine learning could successfully achieve very high accuracy (RNN-LSTM AUROC = 1; decision trees AUROC = 0.954; logistic regression AUROC = 0.728), with RNN-LSTM showing the best results. Further analyses revealed that the change of non-renal Sequential Organ Failure Assessment (SOFA) score between the 1st day and 3rd day (Δnon-renal SOFA) is instrumental in predicting the occurrence of AKD.Conclusion: Our results showed that machine learning, particularly RNN-LSTM, can accurately predict AKD occurrence. In addition, Δ SOFAnon−renal plays an important role in predicting the occurrence of AKD.

show abstract

Economic LSTM Approach for Recurrent Neural Networks

Cited by 59 publications

References 13 publications

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

A Novel GAPG Approach to Automatic Property Generation for Formal Verification: The GAN Perspective

Explicit Duration Recurrent Networks

Application of Machine Learning to Predict Acute Kidney Disease in Patients With Sepsis Associated Acute Kidney Injury

Contact Info

Product

Resources

About