GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models

Prasad, Archiki; Hase, Peter; Zhang, Xiang; Bansal, Mohit

doi:10.48550/arxiv.2203.07281

Cited by 13 publications

(29 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, the optimized prompts, though inducing strong task performance, tend to be gibberish text without clear human-understandable meaning, echoing the recent research (Webson and Pavlick, 2021;Zhao et al, 2021;Prasad et al, 2022) that LMs making use of prompts do not necessarily follow human language patterns. Perhaps surprisingly, those gibberish prompts learned with one LM can be used in other LMs for significant performance, indicating that those different pretrained LMs have grasped shared structures for prompting.…”

Section: Introductionmentioning

confidence: 66%

“…The optimization above, however, can be intractable because the discrete tokens of z are not amenable to gradient-based optimization, while the brute-force search space grows exponentially in the order of O(V L ). Previous work either approximate gradients over z using their continuous LM embeddings (Shin et al, 2020) or tweak human-written prompts with heuristics (Jiang et al, 2020;Mishra et al, 2021a;Prasad et al, 2022), to some success.…”

Section: The Discrete Prompt Optimization Problemmentioning

confidence: 99%

“…Instructions Separate from but related to manual prompts, another line of work (Weller et al, 2020;Efrat and Levy, 2020;Mishra et al, 2021b;Wang et al, 2022) makes use of instructional prompts which provide task descriptions instead of fill-inthe-blank questions. In particular, instruction metatuning (Mishra et al, 2021b;Zhong et al, 2021;Wei et al, 2022a) Discrete Prompt Enumeration Because discrete prompts are difficult to optimize and susceptible to small design variations (Zhao et al, 2021;Webson and Pavlick, 2021;Lu et al, 2021), a number of existing works seek to locate better prompts by augmenting human-written prompts with heuristics such as paraphrasing (Jiang et al, 2020;Gao et al, 2021), editing (Prasad et al, 2022), and reframing (Mishra et al, 2021a). The final prompt is typically selected to maximize some downstream performance metric.…”

Section: Prompting Paradigmsmentioning

confidence: 99%

“…However, the discrete nature of the prompts renders the optimization very difficult. Previous work has typically relied on manual engineering with heuristics (Petroni et al, 2019;Brown et al, 2020;Schick and Schütze, 2021a;Tam et al, 2021), or automatic enumeration of multiple prompt candidates, from which the best one is picked (Jiang et al, 2020;Gao et al, 2021;Liu et al, 2021b;Prasad et al, 2022). AutoPrompt (Shin et al, 2020) uses gradient information to edit the prompt tokens, which suffers from training instability as well as the same applicability issue as gradient-based soft prompting, showing limited Table 1: Comparison of different (prompting) paradigms for using pretrained LMs on downstream tasks, in terms of a number of desirable properties.…”

Section: Introductionmentioning

confidence: 99%

“…Discrete Prompt Enum. enumerates discrete prompts (e.g., by paraphrasing or generation) from which the best is selected (e.g., Jiang et al, 2020;Gao et al, 2021;Liu et al, 2021b;Prasad et al, 2022). AutoPrompt (Shin et al, 2020) uses gradients to edit the discrete prompt tokens.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning

Deng¹,

Wang²,

Hsieh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Prompting has shown impressive success in enabling large pretrained language models (LMs) to perform diverse NLP tasks, especially when only few downstream data are available. Automatically finding the optimal prompt for each task, however, is challenging. Most existing work resorts to tuning soft prompt (e.g., embeddings) which falls short of interpretability, reusability across LMs, and applicability when gradients are not accessible. Discrete prompt, on the other hand, is difficult to optimize, and is often created by "enumeration (e.g., paraphrasing)-then-selection" heuristics that do not explore the prompt space systematically. This paper proposes RLPROMPT, an efficient discrete prompt optimization approach with reinforcement learning (RL). RL-PROMPT formulates a parameter-efficient policy network that generates the desired discrete prompt after training with reward. To overcome the complexity and stochasticity of reward signals by the large LM environment, we incorporate effective reward stabilization that substantially enhances the training efficiency. RLPROMPT is flexibly applicable to different types of LMs, such as masked (e.g., BERT) and left-to-right models (e.g., GPTs), for both classification and generation tasks. Experiments on few-shot classification and unsupervised text style transfer show superior performance over a wide range of existing finetuning or prompting methods. Interestingly, the resulting optimized prompts are often ungrammatical gibberish text; and surprisingly, those gibberish prompts are transferrable between different LMs to retain significant performance, indicating LM prompting may not follow human language patterns. 1

show abstract

Section: Introductionmentioning

confidence: 66%

Section: The Discrete Prompt Optimization Problemmentioning

confidence: 99%

Section: Prompting Paradigmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning

Deng¹,

Wang²,

Hsieh³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning

Li,

Liu,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

Prompt-based pre-trained language models (PLMs) paradigm has succeeded substantially in few-shot natural language processing (NLP) tasks. However, prior discrete prompt optimization methods require expert knowledge to design the base prompt set and identify high-quality prompts, which is costly, inefficient, and subjective. Meanwhile, existing continuous prompt optimization methods improve the performance by learning the ideal prompts through the gradient information of PLMs, whose high computational cost, and low readability and generalizability are often concerning. To address the research gap, we propose a Dialogue-comprised Policy-gradient-based Discrete Prompt Optimization (DP_2O) method. We first design a multi-round dialogue alignment strategy for readability prompt set generation based on GPT-4. Furthermore, we propose an efficient prompt screening metric to identify high-quality prompts with linear complexity. Finally, we construct a reinforcement learning (RL) framework based on policy gradients to match the prompts to inputs optimally. By training a policy network with only 0.62M parameters on the tasks in the few-shot setting, DP_2O outperforms the state-of-the-art (SOTA) method by 1.52% in accuracy on average on four open-source datasets. Moreover, subsequent experiments also demonstrate that DP_2O has good universality, robustness and generalization ability.

show abstract

Do Prompt-Based Models Really Understand the Meaning of Their Prompts?

Webson¹,

Pavlick

2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Recently, a boom of papers has shown extraordinary progress in zero-shot and few-shot learning with various prompt-based models. It is commonly argued that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. In this study, we experiment with over 30 prompt templates manually written for natural language inference (NLI). We find that models can learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading as they do with instructively "good" prompts. Further, such patterns hold even for models as large as 175 billion parameters (Brown et al., 2020) as well as the recently proposed instruction-tuned models which are trained on hundreds of prompts (Sanh et al., 2021). That is, instruction-tuned models often produce good predictions with irrelevant and misleading prompts even at zero shots. In sum, notwithstanding prompt-based models' impressive improvement, we find evidence of serious limitations that question the degree to which such improvement is derived from models understanding task instructions in ways analogous to humans' use of task instructions. * Unabridged version available on arXiv. Code, interactive figures, and statistical test results available at https://github. com/awebson/prompt_semantics arbitrary dimensions of a one-hot vector. In contrast, suppose a human is given a prompt such as: Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that " Given that "no weapons of mass destruction found in Iraq yet.", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that " ", is it definitely correct that "weapons of mass destruction found in Iraq."? "? "? "? "? "? "? "? "? "? "? "? "? "? "? "? "? 1 Then it would be no surprise that they are able to perform the task more accurately and without needing many examples to figure out what the task is.

show abstract

GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models

Cited by 13 publications

References 0 publications

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning

RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning

Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning

Do Prompt-Based Models Really Understand the Meaning of Their Prompts?

Contact Info

Product

Resources

About