AdvDO: Realistic Adversarial Attacks for Trajectory Prediction

Abstract

Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors. While many prior works aim to achieve higher prediction accuracy, few study the adversarial robustness of their methods. To bridge this gap, we propose to study the adversarial robustness of data-driven trajectory prediction systems. We devise an optimization-based adversarial attack framework that leverages a carefully-designed differentiable dynamic model to generate realistic adversarial trajectories. Empirically, we benchmark the adversarial robustness of state-of-the-art prediction models and show that our attack increases the prediction error for both general metrics and planningaware metrics by more than 50% and 37%. We also show that our attack can lead an AV to drive off road or collide into other vehicles in simulation. Finally, we demonstrate how to mitigate the adversarial attacks using an adversarial training scheme 1 .

On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles

“…The objective of trajectory prediction is to model the future trajectory of other road users (vehicles, pedestrians and cyclists), often referred to as agents, taking into account their past states. The state x t i of an agent i usually contains its position and potentially velocity, acceleration, and heading angle [16]. Additionally, environmental contexts, such as road lines and sidewalks, are sometimes considered as well [16].…”

“…Natural robustness of trajectory prediction implies robustness against perturbations that adhere to the physical constraints of actual vehicle driving and reflect normal driving behavior rather than adversarial maneuvers [16], [30]. If generated trajectories appear unnatural to an autonomous vehicle, they may exploit this property to predict an anomaly or an attack [30].…”

“…It includes modifying the training data or augmenting it with perturbations (Figure 1). These perturbations can be incorporated into the historical states X [7], [16], [18], [23], [38] and optionally into the future states Y [14], [24], [39], denoted as X and Ỹ . The objective is for the trained model to exhibit comparable behavior for perturbed and benign examples [40].…”

Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction

“…Recently, Transformer structure [63] is applied in this task [21,62,76,77] to model the spatio-temporal relations via an attention mechanism. Moreover, various viewpoints have emerged towards more practical applications, i.e., goal-driven idea [13,40,60,81], long-tail situation [39], interpretability [32], robustness [9,66,70,80], counterfactual analysis [11], planningdriven [12], generalization ability to new environment [6,27,72], and knowledge distillation [44].…”

Robust Trajectory Tracking for Underactuated Quadrotors with Prescribed Performance

“…Prediction-based navigation in a decentralized event-based scheme is studied in [10]. Moreover, learning-based prediction approaches for multiagent systems in the recent years became very popular area of research [11]- [15]. Unfortunately, most of these approaches require extensive amount of data for the training purpose and direct transferability to our particular application is unknown.…”