Forward to the Past: CASTLab’s Cyber-Social-Physical Approach for ITS in 1999 [History and Perspectives]

Wang, Fei-Yue

doi:10.1109/mits.2023.3278158

Cited by 6 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite demonstrating excellent performance, their operation resembles a "black box", meaning the entire decision process lacks intuitive interpretability and is difficult for humans to understand, leading to widespread public distrust and legal concerns. Moreover, current rule-based and learning-based methods lack a certain degree of social intelligence [17][18][19][20]. They treat autonomous driving as a mechanical task, viewing the interaction between the vehicle and its surroundings as simple kinematic cooperation [21,22], while neglecting the social cognitive context that is crucial for understanding driving behavior.…”

Section: Introductionmentioning

confidence: 99%

DriveLLaVA: Human-Level Behavior Decisions via Vision Language Model

Zhao,

Yuan,

et al. 2024

Sensors

View full text Add to dashboard Cite

Human-level driving is the ultimate goal of autonomous driving. As the top-level decision-making aspect of autonomous driving, behavior decision establishes short-term driving behavior strategies by evaluating road structures, adhering to traffic rules, and analyzing the intentions of other traffic participants. Existing behavior decisions are primarily implemented based on rule-based methods, exhibiting insufficient generalization capabilities when faced with new and unseen driving scenarios. In this paper, we propose a novel behavior decision method that leverages the inherent generalization and commonsense reasoning abilities of visual language models (VLMs) to learn and simulate the behavior decision process in human driving. We constructed a novel instruction-following dataset containing a large number of image–text instructions paired with corresponding driving behavior labels, to support the learning of the Drive Large Language and Vision Assistant (DriveLLaVA) and enhance the transparency and interpretability of the entire decision process. DriveLLaVA is fine-tuned on this dataset using the Low-Rank Adaptation (LoRA) approach, which efficiently optimizes the model parameter count and significantly reduces training costs. We conducted extensive experiments on a large-scale instruction-following dataset, and compared with state-of-the-art methods, DriveLLaVA demonstrated excellent behavior decision performance. DriveLLaVA is capable of handling various complex driving scenarios, showing strong robustness and generalization abilities.

show abstract