Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling

Nottingham, Kolby; Ammanabrolu, Prithviraj; Suhr, Alane; Choi, Yejin; Hajishirzi, Hannaneh; Singh, Sameer; Fox, Roy

doi:10.48550/arxiv.2301.12050

Cited by 4 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, large language models are often combined with agent decision-making, usually introducing the text knowledge of large language models to improve agent training and generalization [32]. In cases of vast state spaces, large language models are used to directly make decisions for agents [33]. Unlike all previous research, in this paper, we focus on using large language models for trajectory selection to enhance the training speed of reinforcement learning agents.…”

Section: Related Workmentioning

confidence: 99%

Sample Trajectory Selection Method Based on Large Language Model in Reinforcement Learning

Lai,

Zang

2024

IEEE Access

View full text Add to dashboard Cite

This paper introduces a method for trajectory selection using large-scale pre-trained language models, aimed at improving sample efficiency and training efficiency in reinforcement learning. By utilizing a carefully designed prompt, we enable the large language model to fully utilize its prior knowledge, effectively understanding and assessing the quality of trajectories produced through agent-environment interactions in reinforcement learning. This approach allows selecting more informative trajectories for the current agent's learning. Unlike other works that use large language models to indirectly improve reinforcement learning training efficiency by generating actions or decisions, our method employs these models to choose superior trajectories, thus more directly enhancing sample efficiency in reinforcement learning. We evaluated this approach on multiple benchmark tasks in OpenAI's Gym and Rlcard. The results indicate a significant reduction in the number of environment interactions, with increasing the average reward by 37% compared to the original method.

show abstract

Section: Related Workmentioning

confidence: 99%

Sample Trajectory Selection Method Based on Large Language Model in Reinforcement Learning

Lai,

Zang

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Dialogue for information gathering as in [160] can accelerate training. Other works such as [106,143] employ autonomous agents for embodied decision-making and exploration guided by internal world models.…”

Section: Engineeringmentioning

confidence: 99%

“…SmolModels [44], DemoGPT [41] Aerospace Engineering IELLM [107] Industrial Automation GPT4IA [144] Robotics & Embodied AI Planner-Actor-Reporter [25], Dialogue Shaping [160], DECKARD [106], TaPA [143],…”

Section: General Autonomous Ai Agentmentioning

confidence: 99%

Genomic Surveillance for SARS-CoV-2 — China, September 26, 2022 to January 29, 2023

Wang¹,

Niu²,

Su³

et al. 2023

China CDC Weekly

View full text Add to dashboard Cite

Introduction: The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has generated 2,431 variants over the course of its global transmission over the past 3 years. To better evaluate the genomic variation of SARS-CoV-2 before and after the optimization of coronavirus disease 2019 (COVID-19) prevention and control strategies, we analyzed the genetic evolution branch composition and genomic variation of SARS-CoV-2 in both domestic and imported cases in China (the data from Hong Kong and Macau Special Administrativc Regions and Taiwan, China were not included) from September 26, 2022 to January 29, 2023.Methods: Analysis of the number of genome sequences, sampling time, dynamic changes of evolutionary branches, origin, and clinical typing of SARS-CoV-2 variants submitted by 31 provincial-level administrative divisions (PLADs) and Xinjiang Production and Construction Corp (XPCC) was conducted to assess the accuracy and timeliness of SARS-CoV-2 variant surveillance.Results: From September 26, 2022 to January 29, 2023, 20,013 valid genome sequences of domestic cases were reported in China, with 72 evolutionary branches. Additionally, 1,978 valid genome sequences of imported cases were reported, with 169 evolutionary branches. The prevalence of the Omicron variants of SARS-CoV-2 in both domestic and imported cases was consistent with that of international epidemic variants.Conclusions: This study provides an overview of the prevalence of Omicron variants of SARS-CoV-2 in China. After optimizing COVID-19 prevention and control strategies, no novel Omicron variants of SARS-CoV-2 with altered biological characteristics or public health significance have been identified since

show abstract

“…• Source of Inspiration, Suggestions: LLMs can serve as an invaluable source of inspiration and suggestions, helping users brainstorm ideas [602], create content [603], and make decisions [604]. • Copilots Over Autonomous Agents: Given its limitations, LLMs are better suited as a 'copilot' that provides assistance and suggestions, rather than an autonomous agent that acts without human input or oversight [605], [606].…”

Section: B Ethical Considerationsmentioning

confidence: 99%

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Hadi,

tashi,

Qureshi

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Within the vast expanse of computerized language processing, a revolutionary entity known as Large Language Models (LLMs) has emerged, wielding immense power in its capacity to comprehend intricate linguistic patterns and conjure coherent and contextually fitting responses. Large language models (LLMs) are a type of artificial intelligence (AI) that have emerged as powerful tools for a wide range of tasks, including natural language processing (NLP), machine translation, and question-answering. This survey paper provides a comprehensive overview of LLMs, including their history, architecture, training methods, applications, and challenges. The paper begins by discussing the fundamental concepts of generative AI and the architecture of generative pre- trained transformers (GPT). It then provides an overview of the history of LLMs, their evolution over time, and the different training methods that have been used to train them. The paper then discusses the wide range of applications of LLMs, including medical, education, finance, and engineering. It also discusses how LLMs are shaping the future of AI and how they can be used to solve real-world problems. The paper then discusses the challenges associated with deploying LLMs in real-world scenarios, including ethical considerations, model biases, interpretability, and computational resource requirements. It also highlights techniques for enhancing the robustness and controllability of LLMs, and addressing bias, fairness, and generation quality issues. Finally, the paper concludes by highlighting the future of LLM research and the challenges that need to be addressed in order to make LLMs more reliable and useful. This survey paper is intended to provide researchers, practitioners, and enthusiasts with a comprehensive understanding of LLMs, their evolution, applications, and challenges. By consolidating the state-of-the-art knowledge in the field, this survey serves as a valuable resource for further advancements in the development and utilization of LLMs for a wide range of real-world applications. The GitHub repo for this project is available at https://github.com/anas-zafar/LLM-Survey</p>

show abstract

Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling

Cited by 4 publications

References 18 publications

Sample Trajectory Selection Method Based on Large Language Model in Reinforcement Learning

Sample Trajectory Selection Method Based on Large Language Model in Reinforcement Learning

Genomic Surveillance for SARS-CoV-2 — China, September 26, 2022 to January 29, 2023

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Contact Info

Product

Resources

About