AEON: a method for automatic evaluation of NLP test cases

Huang, Jen-tse; Zhang, Jianping; Wang, Wenxuan; He, Pinjia; Su, Yuxin; Lyu, Michael R.

doi:10.1145/3533767.3534394

Cited by 14 publications

(5 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By leveraging web-based platforms, educators can create interactive and engaging learning experiences for students, facilitating language acquisition and proficiency. Huang et al (2022) proposed AEON method for automatic evaluation of NLP test cases contributes to the advancement of automated testing techniques in natural language processing. Automated evaluation methods like AEON are essential for ensuring the accuracy and reliability of NLP systems in various applications, including chatbots, virtual assistants, and machine translation.…”

Section: Literature Reviewmentioning

confidence: 99%

“…By considering factors such as word frequency, concreteness, and connotation, these models can provide a more nuanced evaluation of vocabulary usage. The development of large-scale annotated datasets for vocabulary assessment has facilitated the training and evaluation of machine learning models [6]. These datasets contain diverse texts with annotated vocabulary levels, allowing researchers to benchmark the performance of different algorithms and improve their accuracy over time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Algorithm Design and Implementation of Automatic Assessment and Expansion of English Vocabulary

Niu

2024

jes

View full text Add to dashboard Cite

Automatic assessment and expansion of English vocabulary play a crucial role in language learning and proficiency evaluation. In this paper, we propose the use of an Optimized Bi-directional Long Short-Term Memory Deep Learning (Obi-LSTM-DL) model for these purposes. We conduct experiments using a comprehensive vocabulary dataset and evaluate the model's performance across various scenarios. The results demonstrate the effectiveness of the Obi-LSTM-DL model in accurately assessing vocabulary proficiency levels and expanding vocabulary knowledge. Through optimization experiments, we show that increasing the vocabulary size leads to improved model performance. Additionally, the model's proficiency level assessment capability allows for tailored instruction and support for language learners. Comparative analysis with baseline models further confirms the superiority of the Obi-LSTM-DL model. The results demonstrate the effectiveness of the Obi-LSTM-DL model in accurately assessing vocabulary proficiency levels and expanding vocabulary knowledge. Through optimization experiments, we show that increasing the vocabulary size leads to improved model performance, with an average increase in accuracy of 1.5% for every 1,000 words added to the vocabulary. Additionally, the model's proficiency level assessment capability allows for tailored instruction and support for language learners. Comparative analysis with baseline models further confirms the superiority of the Obi-LSTM-DL model, with an average increase in accuracy of 3.2% over traditional models.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Algorithm Design and Implementation of Automatic Assessment and Expansion of English Vocabulary

Niu

2024

jes

View full text Add to dashboard Cite

show abstract

“…NLP software, such as machine translation software and chatbot, has also been widely used in human life. Similar to AI software, researchers have proposed variance methods to validate the reliability of NLP software on the correctness [18,19,40,44], toxicity [53,54], fairness [47,52,57].…”

Section: Related Work 61 Testing Of Ai Softwarementioning

confidence: 99%

Characterization of Two BAHD Acetyltransferases Highly Expressed in the Flowers of Jasminum sambac (L.) Aiton

Wang

Zhang

Chen

et al. 2021

Plants

View full text Add to dashboard Cite

Volatile benzenoid compounds are found in diverse aromatic bouquets emitted by most moth-pollinated flowers. The night-blooming Jasminum sambac is widely cultivated worldwide in the tropics and subtropics for ornamental and industrial purposes owing to its fragrant flowers. Benzylacetate is a characteristic constituent in jasmine scent which makes up to approximately 20–30% of the total emission in the headspace or extract, but the biosynthesis enzymes and the encoding genes have not yet been described. Here, we identify two cytosolic BAHD acyltransferases specifically expressed in the petals with a positive correlation closely to the emission pattern of the volatile benzenoids. Both JsBEAT1 and JsBEAT2 could use benzylalcohol and acetate-CoA as substrates to make benzylacetate in vitro. The recombinant GST-JsBEAT1 has an estimated apparent Km of 447.3 μM for benzylalcohol and 546.0 μM for acetate-CoA, whereas in the instance of the His-JsBEAT2, the Km values are marginally lower, being 278.7 and 317.3 μM, respectively. However, the catalytic reactions by the GST-JsBEAT1 are more efficient than that by the His-JsBEAT2, based on the steady-state kcat parameters. Furthermore, ectopic expression of JsBEAT1 and JsBEAT2 in the transgenic P. hybrida plants, driven by a flower-specific promotor, significantly enhances the biosynthesis of benzylbenzoate and benzylacetate, as well as the total VOCs.

show abstract

“…The existence of adversarial samples causes negative effects on security-sensitive DNNbased applications, such as self-driving [27] and medical diagnosis [6,7]. Therefore, it is necessary to understand the DNNs [15,32,33,40] and enhance attack algorithms to better identify the DNN model's vulnerability, which is the first step to improve their robustness against adversarial samples [23,24,42].…”

Section: Introductionmentioning

confidence: 99%

Improving the Transferability of Adversarial Samples by Path-Augmented Method

Zhang¹,

Huang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep neural networks have achieved unprecedented success on diverse vision tasks. However, they are vulnerable to adversarial noise that is imperceptible to humans. This phenomenon negatively affects their deployment in real-world scenarios, especially security-related ones. To evaluate the robustness of a target model in practice, transfer-based attacks craft adversarial samples with a local model and have attracted increasing attention from researchers due to their high efficiency. The state-of-the-art transfer-based attacks are generally based on data augmentation, which typically augments multiple training images from a linear path when learning adversarial samples. However, such methods selected the image augmentation path heuristically and may augment images that are semantics-inconsistent with the target images, which harms the transferability of the generated adversarial samples. To overcome the pitfall, we propose the Path-Augmented Method (PAM). Specifically, PAM first constructs a candidate augmentation path pool. It then settles the employed augmentation paths during adversarial sample generation with greedy search. Furthermore, to avoid augmenting semantics-inconsistent images, we train a Semantics Predictor (SP) to constrain the length of the augmentation path. Extensive experiments confirm that PAM can achieve an improvement of over 4.8% on average compared with the state-of-the-art baselines in terms of the attack success rates.

show abstract

AEON: a method for automatic evaluation of NLP test cases

Cited by 14 publications

References 50 publications

Algorithm Design and Implementation of Automatic Assessment and Expansion of English Vocabulary

Algorithm Design and Implementation of Automatic Assessment and Expansion of English Vocabulary

Characterization of Two BAHD Acetyltransferases Highly Expressed in the Flowers of Jasminum sambac (L.) Aiton

Improving the Transferability of Adversarial Samples by Path-Augmented Method

Contact Info

Product

Resources

About