Large Language Models for UAVs: Current State and Pathways to the Future

Javaid, Shumaila; Fahim, Hamza; He, Bin; Saeed, Nasir

doi:10.1109/ojvt.2024.3446799

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Other1

Article1

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 172 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Patrol Agent: An Autonomous UAV Framework for Urban Patrol Using on Board Vision Language Model and on Cloud Large Language Model

Yuan,

Xie,

2024

2024 6th International Conference on Robotics and Computer Vision (ICRCV)

View full text Add to dashboard Cite

Patrol Agent: An Autonomous UAV Framework for Urban Patrol Using on Board Vision Language Model and on Cloud Large Language Model

Yuan,

Xie,

2024

2024 6th International Conference on Robotics and Computer Vision (ICRCV)

View full text Add to dashboard Cite

LEVIOSA: Natural Language-Based Uncrewed Aerial Vehicle Trajectory Generation

Aikins,

Dao,

Moukpe

et al. 2024

Electronics

View full text Add to dashboard Cite

This paper presents LEVIOSA, a novel framework for text- and speech-based uncrewed aerial vehicle (UAV) trajectory generation. By leveraging multimodal large language models (LLMs) to interpret natural language commands, the system converts text and audio inputs into executable flight paths for UAV swarms. The approach aims to simplify the complex task of multi-UAV trajectory generation, which has significant applications in fields such as search and rescue, agriculture, infrastructure inspection, and entertainment. The framework involves two key innovations: a multi-critic consensus mechanism to evaluate trajectory quality and a hierarchical prompt structuring for improved task execution. The innovations ensure fidelity to user goals. The framework integrates several multimodal LLMs for high-level planning, converting natural language inputs into 3D waypoints that guide UAV movements and per-UAV low-level controllers to control each UAV in executing its assigned 3D waypoint path based on the high-level plan. The methodology was tested on various trajectory types with promising accuracy, synchronization, and collision avoidance results. The findings pave the way for more intuitive human–robot interactions and advanced multi-UAV coordination.

show abstract

Large Language Models for UAVs: Current State and Pathways to the Future

Cited by 2 publications

References 172 publications

Patrol Agent: An Autonomous UAV Framework for Urban Patrol Using on Board Vision Language Model and on Cloud Large Language Model

Patrol Agent: An Autonomous UAV Framework for Urban Patrol Using on Board Vision Language Model and on Cloud Large Language Model

LEVIOSA: Natural Language-Based Uncrewed Aerial Vehicle Trajectory Generation

Contact Info

Product

Resources

About