Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition

Milani, Stephanie; Kanervisto, Anssi; Ramanauskas, Karolis; Schulhoff, Sander; Houghton, Brandon; Mohanty, Sharada P.; Galbraith, Byron; Chen, Ke; Yan, Shuicheng; Zhou, Tong; Yu, Bingquan; Li, He; Guan, Kai; Hu, Yujing; Tangjie, Lv,; Malato, Federico; Leopold, Florian; Amogh, Raut,; Hautamäki, Ville; Melnik, Andrew; Ishida, Shohei; Henriques, João F.; Klassert, Robert; Laurito, Walter; Novoseller, Ellen; Goecks, Vinicius G.; Waytowich, Nicholas R.; Watkins, David I.; Miller, Josh; Shah, Rohin

doi:10.48550/arxiv.2303.13512

Cited by 1 publication

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, with large-scale web data, VPT [2] builds a foundation model for Minecraft by learning from videos. Based on its success, many works [18] have also explored to finetune foundation model with human feedback. On the other hand, as Minecraft agents become increasingly proficient in handling simple tasks, the importance of multi-task learning becomes more prominent.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Research on university teaching based on artificial intelligence

Zhu¹

2021

Computational Social Science

View full text Add to dashboard Cite

The captivating realm of Minecraft has attracted substantial research interest in recent years, serving as a rich platform for developing intelligent agents capable of functioning in open-world environments. However, the current research landscape predominantly focuses on specific objectives, such as the popular "ObtainDiamond" task, and has not yet shown effective generalization to a broader spectrum of tasks. Furthermore, the current leading success rate for the "ObtainDiamond" task stands at around 20%, highlighting the limitations of Reinforcement Learning (RL) based controllers used in existing methods. To tackle these challenges, we introduce Ghost in the Minecraft (GITM), a novel framework integrates Large Language Models (LLMs) with text-based knowledge and memory, aiming to create Generally Capable Agents (GCAs) in Minecraft. These agents, equipped with the logic and common sense capabilities of LLMs, can skillfully navigate complex, sparse-reward environments with text-based interactions. We develop a set of structured actions and leverage LLMs to generate action plans for the agents to execute. The resulting LLM-based agent markedly surpasses previous methods, achieving a remarkable improvement of +47.5% in success rate on the "ObtainDiamond" task, demonstrating superior robustness compared to traditional RL-based controllers. Notably, our agent is the first to procure all items in the Minecraft Overworld technology tree, demonstrating its extensive capabilities. GITM does not need any GPU for training, but a single CPU node with 32 CPU cores is enough. This research shows the potential of LLMs in developing capable agents for handling long-horizon, complex tasks and adapting to uncertainties in open-world environments. See the project website at https://github.com/ OpenGVLab/GITM.

show abstract

Section: Related Workmentioning

confidence: 99%

“…However, existing researches [2,7,25] remain narrowly scoped. Prior studies have predominantly focused on the specific goal of ObtainDiamond [18]. Yet, in the process of obtaining diamonds, the number of types of items involved only accounts for <5% of the entire Minecraft world.…”

Section: Introductionmentioning

confidence: 99%

Research on university teaching based on artificial intelligence

Zhu¹

2021

Computational Social Science

View full text Add to dashboard Cite

show abstract

Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition

Cited by 1 publication

References 17 publications

Research on university teaching based on artificial intelligence

Research on university teaching based on artificial intelligence

Contact Info

Product

Resources

About