Measure and Improve Robustness in NLP Models: A Survey

Wang, Xuezhi; Wang, Haohan; Yang, Diyi

doi:10.48550/arxiv.2112.08313

Cited by 8 publications

(9 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…typos) that cause most current systems to significantly degrade (Szegedy et al, 2014;Goodfellow et al, 2015;Jia and Liang, 2017;Belinkov and Bisk, 2018;Madry et al, 2018;Ribeiro et al, 2020;Santurkar et al, 2020;Tsipras, 2021;. Thus, in order to better capture the performance of these models in practice, we need to expand our evaluation beyond the exact instances contained in our scenarios (Jia and Liang, 2017;Goel et al, 2021;Wang et al, 2021b).…”

Section: Robustnessmentioning

confidence: 99%

Holistic Evaluation of Language Models

Liang¹,

Bommasani²,

Lee³

et al. 2022

Preprint

View full text Add to dashboard Cite

Language models (LMs) are becoming the foundation for almost all major language technologies, but their capabilities, limitations, and risks are not well understood. We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models. First, we taxonomize the vast space of potential scenarios (i.e. use cases) and metrics (i.e. desiderata) that are of interest for LMs. Then we select a broad subset based on coverage and feasibility, noting what's missing or underrepresented (e.g. question answering for neglected English dialects, metrics for trustworthiness). Second, we adopt a multi-metric approach: We measure 7 metrics (accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency) for each of 16 core scenarios to the extent possible (87.5% of the time), ensuring that metrics beyond accuracy don't fall to the wayside, and that trade-offs across models and metrics are clearly exposed. We also perform 7 targeted evaluations, based on 26 targeted scenarios, to more deeply analyze specific aspects (e.g. knowledge, reasoning, memorization/copyright, disinformation). Third, we conduct a large-scale evaluation of 30 prominent language models (spanning open, limited-access, and closed models) on all 42 scenarios, including 21 scenarios that were not previously used in mainstream LM evaluation. Prior to HELM, models on average were evaluated on just 17.9% of the core HELM scenarios, with some prominent models not sharing a single scenario in common. We improve this to 96.0%: now all 30 models have been densely benchmarked on a set of core scenarios and metrics under standardized conditions. Our evaluation surfaces 25 top-level findings concerning the interplay between different scenarios, metrics, and models. For full transparency, we release all raw model prompts and completions publicly 3 for further analysis, as well as a general modular toolkit for easily adding new scenarios, models, metrics, and prompting strategies. 4 We intend for HELM to be a living benchmark for the community, continuously updated with new scenarios, metrics, and models.

show abstract

Section: Robustnessmentioning

confidence: 99%

Holistic Evaluation of Language Models

Liang¹,

Bommasani²,

Lee³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The NLP community has also examined voice assistant failures from a slightly different angle, focusing on the robustness of different NLP components underlying voice assistants, such as models for tasks in natural language inference [44], question answering [26,42], and speech recognition [32]. NLP robustness can be defined as understanding how model performance changes when testing on a new dataset, which has a different distribution from the dataset the model is trained on [62]. In practice, users' real world interactions with voice assistants could differ from data used in development, which mimics the data distribution shift in NLP robustness research.…”

Section: Nlp Approaches To Voice Assistant Failuresmentioning

confidence: 99%

A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures

Baughan

Mercurio

Liu

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Despite huge gains in performance in natural language understanding via large language models in recent years, voice assistants still often fail to meet user expectations. In this study, we conducted a mixed-methods analysis of how voice assistant failures affect users' trust in their voice assistants. To illustrate how users have experienced these failures, we contribute a crowdsourced dataset of 199 voice assistant failures, categorized across 12 failure sources.Relying on interview and survey data, we find that certain failures, such as those due to overcapturing users' input, derail user trust more than others. We additionally examine how failures impact users' willingness to rely on voice assistants for future tasks. Users often stop using their voice assistants for specific tasks that result in failures for a short period of time before resuming similar usage. We demonstrate the importance of low stakes tasks, such as playing music, towards building trust after failures. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI.

show abstract

“…Other works inspired by Hendrycks et al [2021b] focus on distribution shift based on changes of grammar errors, dialects, speakers, and language Demszky et al [2020], different domains Miller et al [2020] and bias De-Arteaga et al [2019], Prates et al [2020]. The image robustness research space has inspired many of these studies, but there are vast differences to NLP and vision that make these transfers difficult, such as the discrete vs. continuous search space as explained in Wang et al [2021b]. Data augmentation has also been looked at as a method to improve robustness and has shown substantial improvements Feng et al [2021], Dhole et al [2021], , , Chen et al [2021].…”

Section: Robustnessmentioning

confidence: 99%

Multi-modal Robustness Analysis Against Language and Visual Perturbations

Schiappa¹,

Vyas²,

Palangi³

et al. 2022

Preprint

View full text Add to dashboard Cite

Joint visual and language modeling on large-scale datasets has recently shown a good progress in multi-modal tasks when compared to single modal learning. However, robustness of these approaches against real-world perturbations has not been studied. In this work, we perform the first extensive robustness study of such models against various real-world perturbations focusing on video and language. We focus on text-to-video retrieval and propose two large-scale benchmark datasets, MSRVTT-P and YouCook2-P, which utilize 90 different visual and 35 different textual perturbations. The study reveals some interesting findings: 1) The studied models are more robust when text is perturbed versus when video is perturbed 2) The transformer text encoder is more robust on non-semantic changing text perturbations and visual perturbations compared to word embedding approaches. 3) Using two-branch encoders in isolation is typically more robust than when architectures use cross-attention. We hope this study will serve as a benchmark and guide future research in robust multimodal learning 2 .

show abstract

Measure and Improve Robustness in NLP Models: A Survey

Cited by 8 publications

References 71 publications

Holistic Evaluation of Language Models

Holistic Evaluation of Language Models

A Mixed-Methods Approach to Understanding User Trust after Voice Assistant Failures

Multi-modal Robustness Analysis Against Language and Visual Perturbations

Contact Info

Product

Resources

About