Craft an Iron Sword: Dynamically Generating Interactive Game Characters by Prompting Large Language Models Tuned on Code

Abstract

Non-Player Characters (NPCs) significantly enhance the player experience in many games. Historically, players' interactions with NPCs have tended to be highly scripted, to be limited to natural language responses to be selected by the player, and to not involve dynamic change in game state. In this work, we demonstrate that use of a few example conversational prompts can power a conversational agent to generate both natural language and novel code. This approach can permit development of NPCs with which players can have grounded conversations that are free-form and less repetitive. We demonstrate our approach using OpenAI Codex (GPT-3 finetuned on GitHub), with Minecraft game development as our test bed. We show that with a few example prompts 1 , a Codexbased agent can generate novel code, hold multi-turn conversations and answer questions about structured data. We evaluate this application using experienced gamers in a Minecraft realm and provide analysis of failure cases and suggest possible directions for solutions.

Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data

“…Even for professional developers, it is challenging to create well-designed conversational flows and pre-define user intents and chatbot messages [66]. Using LLMs to power chatbots is a new way to build chatbots [14,94]. LLMs accept natural language prompts so that people without any knowledge of programming but are interested in building chatbots for data collection can create prompts [37].…”

“…OpenAI's website provides an example prompt 1 for open-domain chatbots. For task-oriented chatbots, Volum et al [94] built interactive Non-Player Characters (NPCs) through few-shot prompting. By appending every user input to the initial prompt, the resulted new prompt is fed to the Codex model to generate responses in both code & language.…”

“…Taking prompts that include the conversation history, LLMs can generate the following responses accordingly without any training data, thereby functioning as a chatbot. Compared to other frameworks, LLMs show great potential in scaffolding chatbots that are sensible of contexts and even respond to off-topic user messages [94]. Further, since LLMs operate on natural language inputs, people can have the opportunity to personalize or even build their own chatbots.…”

“…Despite these potentials, it is yet fully understood how LLMs read the prompt and use pretrained knowledge [14,59], the development of prompts is usually conducted through iterative trial and error [61]. While the HCI community have actively explored the use of LLMs in various domains (e.g., [20,49,104]), research that leverages LLMs for powering chatbots, particularly task-oriented ones [8,67,94], is still sparse. Due to the inherent characteristics of LLMs, LLM-driven chatbots may be error-prone [44] or digress from their tasks [94].…”

“…While the HCI community have actively explored the use of LLMs in various domains (e.g., [20,49,104]), research that leverages LLMs for powering chatbots, particularly task-oriented ones [8,67,94], is still sparse. Due to the inherent characteristics of LLMs, LLM-driven chatbots may be error-prone [44] or digress from their tasks [94]. Designing robust prompts is crucial for "restricting" chatbots to conduct desired tasks.…”