Auditing Large Language Models: A Three-Layered Approach

Mökander, Jakob; Schuett, Jonas; Kirk, Hannah Rose; Floridi, Luciano

doi:10.2139/ssrn.4361607

Cited by 29 publications

(28 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is a broad range of ways that LLMs can be customised or adapted to specific use-cases. The paradigm upon which LLMs are built is designed for adaption via transfer learning -where models are first pre-trained, then adapted via fine-tuning or in-context demonstrations for a specific task [161]. Some recent work suggests LLMs require no additional training to 'role-play' as different individuals, adopting their worldview [9], mirroring their play in economic games [91,5] or predicting their voting preferences [11].…”

Section: Customisation Of Llms Already Happensmentioning

confidence: 99%

“…Machine learning workflows affect personalisation as, in principle, every actor involved in creating an LLM application could exert control over how it is adapted or personalised [161]. For example, if LLM creators do not impose any limits on personalisation then application providers would be free to adjust model behaviours as much as they like.…”

Section: How People Interact With Llmsmentioning

confidence: 99%

“…It is harder to assess whether an LLM is "fair" or "harmful" to its user when the space for possible personalisation actions is so complex. These issues also apply to LLMs in general, where adaptation of pre-trained models to downstream applications pose significant challenges to traditional auditing approaches [161].…”

Section: Policy Framework Enforcementmentioning

confidence: 99%

“…In reality, with models like LLaMA [152] being open-sourced, anyone with sufficient technical skill and compute resources could hypo-thetically create, launch and host a system capable of personalisation. Such fragmentation of the development landscape in LLMs poses a challenge for audit and policy enforcement [161].…”

Section: Policy Framework Enforcementmentioning

confidence: 99%

See 3 more Smart Citations

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Kirk¹,

Vidgen²,

Röttger³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large language models (LLMs) are used to generate content for an increasingly wide range of tasks, and are set to reach a growing audience in coming years due to integration in product interfaces like ChatGPT or search engines like Bing. This intensifies the need to ensure that models are aligned with human preferences and do not produce unsafe, inaccurate or toxic outputs. While alignment techniques like reinforcement learning with human feedback (RLHF) and red-teaming can mitigate some safety concerns and improve model capabilities, it is unlikely that an aggregate fine-tuning process can adequately represent the full range of users' preferences and values. Different people may legitimately disagree on their preferences for language and conversational norms, as well as on values or ideologies which guide their communication. Personalising LLMs through micro-level preference learning processes may result in models that are better aligned with each user. However, there are several normative challenges in defining the bounds of a societally-acceptable and safe degree of personalisation. In this paper, we ask how, and in what ways, LLMs should be personalised. First, we review literature on current paradigms for aligning LLMs with human feedback, and identify issues including (i) a lack of clarity regarding what alignment means; (ii) a tendency of technology providers to prescribe definitions of inherently subjective preferences and values; and (iii) a "tyranny of the crowdworker", exacerbated by a lack of documentation in who we are really aligning to. Second, we present a taxonomy of benefits and risks associated with personalised LLMs, for individuals and society at large. Finally, we propose a three-tiered policy framework that allows users to experience the benefits of personalised alignment, while restraining unsafe and undesirable LLM-behaviours within (supra-)national and organisational bounds.

show abstract

Section: Customisation Of Llms Already Happensmentioning

confidence: 99%

Section: How People Interact With Llmsmentioning

confidence: 99%

Section: Policy Framework Enforcementmentioning

confidence: 99%

Section: Policy Framework Enforcementmentioning

confidence: 99%

See 2 more Smart Citations

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Kirk¹,

Vidgen²,

Röttger³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…External model audit, i.e. model evaluation by an independent, external auditor for the purpose of providing a judgement -or input to a judgement -about the safety of deploying a model (or training a new one) (ARC Evals, 2023; Mökander et al, 2023;Raji et al, 2022b).…”

Section: Model Evaluation As Critical Governance Infrastructurementioning

confidence: 99%

Structured Access

Shevlane

2022

The Oxford Handbook of AI Governance

View full text Add to dashboard Cite

Structured access is an emerging paradigm for the safe deployment of artificial intelligence (AI). Instead of openly disseminating AI systems, developers facilitate controlled, arm’s length interactions with their AI systems. The aim is to prevent dangerous AI capabilities from being widely accessible, whilst preserving access to AI capabilities that can be used safely. The developer must both restrict how the AI system can be used and prevent the user from circumventing these restrictions through modification or reverse engineering of the AI system. Structured access is most effective when implemented through cloud-based AI services, rather than disseminating AI software that runs locally on users’ hardware. Cloud-based interfaces provide the AI developer greater scope for controlling how the AI system is used, and for protecting against unauthorized modifications to the system’s design. This chapter expands the discussion of “publication norms” in the AI community, which to date has focused on the question of how the informational content of AI research projects should be disseminated (e.g., code and models). Although this is an important question, there are limits to what can be achieved through the control of information flows. Structured access views AI software not only as information that can be shared but also as a tool with which users can have arm’s length interactions. There are early examples of structured access being practiced by AI developers, but there is much room for further development, both in the functionality of cloud-based interfaces and in the wider institutional framework.

show abstract

Breaking Bad: Unraveling Influences and Risks of User Inputs to ChatGPT for Game Story Generation

Taveekitworachai,

Abdullah,

Gursesli

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This study presents an investigation into the influence and potential risks of using user inputs as part of a prompt, a message used to interact with ChatGPT. We demonstrate the influence of user inputs in a prompt through game story generation and story ending classification. To assess risks, we utilize a technique called adversarial prompting, which involves deliberately manipulating the prompt or parts of the prompt to exploit the safety mechanisms of large language models, leading to undesirable or harmful responses. We assess the influence of positive and negative sentiment words, as proxies for user inputs in a prompt, on the generated story endings. The results suggest that ChatGPT tends to adhere to its guidelines, providing safe and non-harmful outcomes, i.e., positive endings. However, malicious intentions, such as "jailbreaking", can be achieved through prompting injection. These actions carry significant risks of producing unethical outcomes, as shown in an example. As a result, this study also suggests preliminary ways to mitigate these risks: content filtering, rare token-separators, and enhancing training datasets and alignment processes.

show abstract

Auditing Large Language Models: A Three-Layered Approach

Cited by 29 publications

References 0 publications

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Structured Access

Breaking Bad: Unraveling Influences and Risks of User Inputs to ChatGPT for Game Story Generation

Contact Info

Product

Resources

About