Doppelgänger Test Generation for Revealing Bugs in Autonomous Driving Software

Huai, Yuqi; Chen, Yuntianyi; Almanee, Sumaya; Ngo, Tuan; Liao, Xiang; Wan, Ziwen; Alfred, Chen, Qi; Garcia, Joshua

doi:10.1109/icse48619.2023.00216

Cited by 10 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, the more popular method is fuzz testing based on multi-objective optimization (Huai et al, 2023;Cheng et al, 2023;Kim et al, 2022;Li et al, 2020;Tian et al, 2022). The general process of such fuzz testing is as follows:…”

Section: Ads Testing Based On Violated Scenarios Searchmentioning

confidence: 99%

“…Previous works such as Doppelgänger Test (Huai et al, 2023) used the time to control the timing of vehicle entries, which might result in inconsistencies in interactions between NPCs and the ego vehicle due to slight changes in the ego vehicle's behavior like congestion or accelerated driving. TM-fuzzer mitigates inconsistencies by dynamically controlling seed-specified vehicles according to analyzed traffic conditions.…”

Section: Npc Pack and Unpackmentioning

confidence: 99%

“…Hou et al (2023) proposed a method for creating and validating Twin scenarios within a digital twin system. Priisalu et al (2022) introduced a pedestrian movement model framework, and Huai et al (2023)'s DoppelTest innovatively controlled all vehicles in a scenario with the tested ADS. However, the exploration space and effectiveness largely depend on the initial seed scenarios, and, compared to TM-fuzzer, short-term simulation of individual scenarios may overlook certain ADS violations that require longer operational times, as well as global planning flaws.…”

Section: Search-based Testingmentioning

confidence: 99%

“…At present, the behavior of the NPC in current search-based testing methods is inflexible; regardless of the decisions made by the ADS or the roads it traverses, the NPC invariably appears at a predetermined location and follows a specific trajectory. Huai et al (2023) employ intelligent NPCs but the timing of their appearance remains constant. However, if the scenario undergoes mutation and the driving trajectory of the AV remains unchanged, this mutation contributes minimally to the scenario's expansion.…”

Section: Motivationmentioning

confidence: 99%

“…In this context, researchers have proposed many violated scenario search methods (Han and Zhou, 2020;Haq et al, 2022;Zhou et al, 2023a,b), such as guided searching methods (Li et al, 2020;Cheng et al, 2023;Tang et al, 2023), data-based methods Li et al (2022); Hou et al (2023); Priisalu et al (2022), agent-based methods (Huai et al, 2023;Kim et al, 2022;Feng et al, 2023). These methods involve identifying security-critical scenarios within an infinite scenario space that are likely to result in violations, through various risk assessment indicators (e.g., vehicle distance) (Li et al, 2020), and conducting ADS simulation tests.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Lin,

Chen,

et al. 2024

Preprint

View full text Add to dashboard Cite

Simulation testing of Autonomous Driving Systems (ADS) is crucial for ensuring the safety of autonomous vehicles. Currently, scenarios searched by ADS simulation testing tools are less likely to expose ADS issues and highly similar. In this paper, we propose TM-fuzzer, a novel approach for searching ADS test scenarios, which utilizes real-time traffic management and diversity analysis to search security-critical and unique scenarios within the infinite scenario space. TM-fuzzer dynamically manages traffic flow by manipulating non-player characters near autonomous vehicle throughout the simulation process to enhance the efficiency of test scenarios. Additionally, the TM-fuzzer utilizes clustering analysis on vehicle trajectory graphs within scenarios to increase the diversity of test scenarios. Compared to the baseline, the TM-fuzzer identified 29 unique violated scenarios more than four times faster and enhanced the incidence of ADS-caused violations by 26.26%. Experiments suggest that the TM-fuzzer demonstrates improved efficiency and accuracy.

show abstract

Section: Ads Testing Based On Violated Scenarios Searchmentioning

confidence: 99%

Section: Npc Pack and Unpackmentioning

confidence: 99%

Section: Search-based Testingmentioning

confidence: 99%

Section: Motivationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Lin,

Chen,

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Lin,

Chen,

et al. 2024

Autom Softw Eng

View full text Add to dashboard Cite

Collaboration failure analysis in cyber-physical system-of-systems using context fuzzy clustering

Hyun,

Jee,

Bae

2024

Empir Software Eng

View full text Add to dashboard Cite

A cyber-physical system-of-systems (CPSoS) facilitates the achievement of high-level goals, such as efficient traffic management on roads, by designing and developing the collaboration of constituent CPSs. A platooning that groups autonomous vehicles in proximity is an example of collaboration. The intricate collaboration innately causes serious collaboration failures such as collisions. However, limited knowledge and complex dynamics of CPSoS cause several challenges in effectively analyzing the collaboration failures. Existing studies have applied pattern mining techniques to investigate various failures but have limitations when applied to collaboration failures: (1) absence of data model for continuous and discrete logs in CPSoS; (2) information loss problem by not considering the integrated relationship of the data; (3) dependence only on failed logs; (4) limited capability of fixed-size time windows. We propose a fuzzy clustering-based pattern mining approach that consists of a novel data model for CPSoS logs and comprehensive metrics for classifying and mining optimal collaboration failure patterns. In experiments on vehicle platooning, our approach exhibited the highest accuracy on pattern mining and clustering results. Further, we identified five collaboration failure scenarios in the empirical analysis of drone swarming results. The findings of this study can facilitate the effective analysis of CPSoS collaboration failures.

show abstract

Doppelgänger Test Generation for Revealing Bugs in Autonomous Driving Software

Cited by 10 publications

References 41 publications

TM-fuzzer: fuzzing autonomous driving systems through traffic management

TM-fuzzer: fuzzing autonomous driving systems through traffic management

TM-fuzzer: fuzzing autonomous driving systems through traffic management

Collaboration failure analysis in cyber-physical system-of-systems using context fuzzy clustering

Contact Info

Product

Resources

About