SoK: On the Semantic AI Security in Autonomous Driving

Abstract

Autonomous Driving (AD) systems rely on AI components to make safety and correct driving decisions. Unfortunately, today's AI algorithms are known to be generally vulnerable to adversarial attacks. However, for such AI component-level vulnerabilities to be semantically impactful at the system level, it needs to address non-trivial semantic gaps both (1) from the system-level attack input spaces to those at AI component level, and (2) from AI component-level attack impacts to those at the system level. In this paper, we define such research space as semantic AI security as opposed to generic AI security. Over the past 5 years, increasingly more research works are performed to tackle such semantic AI security challenges in AD context, which has started to show an exponential growth trend. However, to the best of our knowledge, so far there is no comprehensive systematization of this emerging research space.In this paper, we thus perform the first systematization of knowledge of such growing semantic AD AI security research space. In total, we collect and analyze 53 such papers, and systematically taxonomize them based on research aspects critical for the security field such as the attack/defense targeted AI component, attack/defense goal, attack vector, attack knowledge, defense deployability, defense robustness, and evaluation methodologies. We summarize 6 most substantial scientific gaps observed based on quantitative comparisons both vertically among existing AD AI security works and horizontally with security works from closely-related domains. With these, we are able to provide insights and potential future directions not only at the design level, but also at the research goal, methodology, and community levels. To address the most critical scientific methodology-level gap, we take the initiative to develop an open-source, uniform, and extensible system-driven evaluation platform, named PASS, for the semantic AD AI security research community. We also use our implemented platform prototype to showcase the capabilities and benefits of such a platform using representative semantic AD AI attacks.

Towards Robustness Analysis of E-Commerce Ranking System

“…Notably, the literature has not thoroughly evaluated the robustness of commercial e-commerce ranking systems or delineated the distinction between real-world and manipulated queries. Commercial ranking systems, such as Google or Amazon, often exhibit greater robustness than standalone ML models [12,44,53]. For example, these systems usually include query rewrite, auto-completion, and search facet to create a smooth search journey, which can potentially avoid the robustness problem from tweaking characters [27] in the input.…”

“…Thus, recent research has focused on ranking robustness by generating adversarial examples to evaluate the model's response disparities [15,28,30,31]. However, these works have limitations: ignore evaluation in commercialized ranking systems, which is generally more robust than ML models [12,44,53], and fail to substantiate the relevance between realworld and manipulated queries. To overcome the first limitation above, we perform the first systematic investigation into the robustness of a leading commercialized e-commerce ranking system.…”

PASS: A System-Driven Evaluation Platform for Autonomous Driving Safety and Security

“…However, we found that system-level evaluation is generally lacking in existing works. Specifically, one of our ongoing works analyzes recent AI security and safety works [36] aimed at creating system-level impact on AD systems in recent 5 years published in commonly-recognized top-tier venues [2] in closely-related fields to AD AI (i.e., security, Computer Vision (CV), Machine Learning (ML), AI, and robotics), as well as a few well-known works published in arXiv and other venues based on our best knowledge. Particularly, for the top-tier venues, we exhaustively search over the paper lists from 2017 to 2021 to find the ones that fall into our scope above.…”

SlowTrack: Increasing the Latency of Camera-Based Perception in Autonomous Driving Using Adversarial Examples

“…For these vehicles, camera-based perception is pivotal, enabling them to detect real-time environmental objects such as pedestrians to ensure safety. Given its significance for safety and security, various prior works (Cao et al 2021;Shen et al 2022;Sato et al 2021a;Wang et al 2023) have studied its security, especially on integrity such as making the object vanished or changing the label of the objects to cause traffic rule violations or safety hazards. We refer to these as system-level effects throughout this paper.…”

“…While some existing AD security analysis has studied availability in object detection (Shapira et al 2023;Chen et al 2023), they do not encompass the entire AD perception since usually, object detection is a part of the AD perception (Jia et al 2020). In addition, in the Cyber-Physical System area, it is widely recognized that small component level errors do not necessarily lead to systemlevel effects (Shen et al 2022;Wang et al 2023). Thus, these studies leave a critical research gap: their proposed attack strategies may not be effective enough to conduct systemlevel effects in end-to-end AD systems.…”