RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems

Guo, Jun; Zhang, Quan; Zhao, Yiqiang; Shi, Heyuan; Jiang, Yu; Sun, Jun

doi:10.1109/tse.2021.3114353

Cited by 20 publications

(16 citation statements)

References 57 publications

(62 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, researchers have proposed diverse variants of neuron coverage as testing criteria focusing on different activation magnitudes [44,79]. Researchers also develop neuron coverage specially designed for recurrent neuron networks [16,23,29] to adopt the properties of sequence inputs. Different from neuron coverage metrics, which often act as test adequacy criteria of the test suite, our approach focuses on the quality evaluation of every test case.…”

Section: Testing Criteriamentioning

confidence: 99%

AEON: A Method for Automatic Evaluation of NLP Test Cases

Huang¹,

Zhang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Due to the labor-intensive nature of manual test oracle construction, various automated testing techniques have been proposed to enhance the reliability of Natural Language Processing (NLP) software. In theory, these techniques mutate an existing test case (e.g., a sentence with its label) and assume the generated one preserves an equivalent or similar semantic meaning and thus, the same label. However, in practice, many of the generated test cases fail to preserve similar semantic meaning and are unnatural (e.g., grammar errors), which leads to a high false alarm rate and unnatural test cases. Our evaluation study finds that 44% of the test cases generated by the state-of-the-art (SOTA) approaches are false alarms. These test cases require extensive manual checking effort, and instead of improving NLP software, they can even degrade NLP software when utilized in model training. To address this problem, we propose AEON for Automatic Evaluation Of NLP test cases. For each generated test case, it outputs scores based on semantic similarity and language naturalness. We employ AEON to evaluate test cases generated by four popular testing techniques on five datasets across three typical NLP tasks. The results show that AEON aligns the best with human judgment. In particular, AEON achieves the best average precision in detecting semantic inconsistent test cases, outperforming the best baseline metric by 10%. In addition, AEON also has the highest average precision of finding unnatural test cases, surpassing the baselines by more than 15%. Moreover, model training with test cases prioritized by AEON leads to models that are more accurate and robust, demonstrating AEON's potential in improving NLP software. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

show abstract

Section: Testing Criteriamentioning

confidence: 99%

AEON: A Method for Automatic Evaluation of NLP Test Cases

Huang¹,

Zhang²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The above-mentioned issue (i) corresponds to the requirement of robustness against input perturbations, also called adversarial attacks or matched disturbances. Ideally, to certify this property one should test the RNN model's response to a sufficiently large amount of perturbed input trajectories [69,70].…”

Section: Robustnessmentioning

confidence: 99%

On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments

Bonassi,

Farina,

Xie

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper aims to discuss and analyze the potentialities of Recurrent Neural Networks (RNN) in control design applications. The main families of RNN are considered, namely Neural Nonlinear AutoRegressive eXogenous, (NNARX), Echo State Networks (ESN), Long Short Term Memory (LSTM), and Gated Recurrent Units (GRU). The goal is twofold. Firstly, to survey recent results concerning the training of RNN that enjoy Input-to-State Stability (ISS) and Incremental Input-to-State Stability (δISS) guarantees. Secondly, to discuss the issues that still hinder the widespread use of RNN for control, namely their robustness, verifiability, and interpretability. The former properties are related to the so-called generalization capabilities of the networks, i.e. their consistency with the underlying real plants, even in presence of unseen or perturbed input trajectories. The latter is instead related to the possibility of providing a clear formal connection between the RNN model and the plant. In this context, we illustrate how ISS and δISS represent a significant step towards the robustness and verifiability of the RNN models, while the requirement of interpretability paves the way to the use of physics-based networks. The design of model predictive controllers with RNN as plant's model is also briefly discussed. Lastly, some of the main topics of the paper are illustrated on a simulated chemical system.

show abstract

“…Current testing techniques for RNNs are limited, leaving a threat to a large number of sequential application domains. To bridge such a gap and improve the generalization performance and robustness of RNNs, custom coverage criteria such as Hidden State Coverage (HS_C) [ 28 ], Boundary Coverage (BC), Stepwise Coverage (SC), and Temporal Coverage (TC) [ 29 ] have been proposed to guide the testing by Guo et al [ 28 ], Du et al [ 30 ], Huang et al [ 29 ]. However, what these researchers have not considered is that the underlying structure of RNNs is still a fully connected neural network with an additional virtual unit “delayer” to record the network state at the time of data input, and we can still use the neurons and weights as the minimum unit of RNNs.…”

Section: Introductionmentioning

confidence: 99%

“…When a contribution is larger than the threshold we specified, it indicates that the contribution is activated. We form a corresponding test framework prototype RNNCon-Test, which outperforms RNN-Test [ 28 ]. Unlike the study by Zhou et al, we do not generate perturbations superimposed on the test inputs by jointly maximizing the contributions and the outputs of the contribution-connected neurons.…”

Section: Introductionmentioning

confidence: 99%

“…Most of the generated test inputs can make the predictions of the corresponding models wrong. We design and implement a test framework prototype RNNCon-Test which outperforms RNN-test [ 28 ]. It is not only effective in detecting defects and improving the performance of Stacked RNNs but can also be extended to variants of RNNs, i.e., LSTMs and GRUs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RNNCon: Contribution Coverage Testing for Stacked Recurrent Neural Networks

Zeng

Chen

et al. 2023

Entropy

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) are applied in safety-critical fields such as autonomous driving, aircraft collision detection, and smart credit. They are highly susceptible to input perturbations, but little research on RNN-oriented testing techniques has been conducted, leaving a threat to a large number of sequential application domains. To address these gaps, improve the test adequacy of RNNs, find more defects, and improve the performance of RNNs models and their robustness to input perturbations. We aim to propose a test coverage metric for the underlying structure of RNNs, which is used to guide the generation of test inputs to test RNNs. Although coverage metrics have been proposed for RNNs, such as the hidden state coverage in RNN-Test, they ignore the fact that the underlying structure of RNNs is still a fully connected neural network but with an additional “delayer” that records the network state at the time of data input. We use the contributions, i.e., the combination of the outputs of neurons and the weights they emit, as the minimum computational unit of RNNs to explore the finer-grained logical structure inside the recurrent cells. Compared to existing coverage metrics, our research covers the decision mechanism of RNNs in more detail and is more likely to generate more adversarial samples and discover more flaws in the model. In this paper, we redefine the contribution coverage metric applicable to Stacked LSTMs and Stacked GRUs by considering the joint effect of neurons and weights in the underlying structure of the neural network. We propose a new coverage metric, RNNCon, which can be used to guide the generation of adversarial test inputs. And we design and implement a test framework prototype RNNCon-Test. 2 datasets, 4 LSTM models, and 4 GRU models are used to verify the effectiveness of RNNCon-Test. Compared to the current state-of-the-art study RNN-Test, RNNCon can cover a deeper decision logic of RNNs. RNNCon-Test is not only effective in identifying defects in Deep Learning (DL) systems but also in improving the performance of the model if the adversarial inputs generated by RNNCon-Test are filtered and added to the training set to retrain the model. In the case where the accuracy of the model is already high, RNNCon-Test is still able to improve the accuracy of the model by up to 0.45%.

show abstract

RNN-Test: Towards Adversarial Testing for Recurrent Neural Network Systems

Cited by 20 publications

References 57 publications

AEON: A Method for Automatic Evaluation of NLP Test Cases

AEON: A Method for Automatic Evaluation of NLP Test Cases

On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments

RNNCon: Contribution Coverage Testing for Stacked Recurrent Neural Networks

Contact Info

Product

Resources

About