Automatic Test and Evaluation of Autonomous Systems

Karimoddini, Ali; Khan, Mubbashar Altaf; Gebreyohannes, Solomon; Heiges, Mike; Trewhitt, Ethan; Homaifar, Abdollah

doi:10.1109/access.2022.3183145

Cited by 7 publications

(4 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To minimize the safety risk, in parallel to the AV deployment efforts, techniques to test and evaluate (T&E)the AVs and their autonomy needed to be enhanced. 103,104 In addition to T&E efforts, fault detection 105,106 and fault accommodation 107,108 techniques needed to be developed to minimize the impact of failures of components of CAVs. Finally, depending on the autonomy level of the AVs and their interactions with road users, the efforts on human-machine interaction research community [109][110][111] should be leveraged to address the challenges of interactions of human-CAVs interactions.…”

Section: Future Challenges For Development and Deployment Of Cavsmentioning

confidence: 99%

“…Therefore, it is crucial to establish comprehensive risk management plans that can adapt to changing circumstances, continuously evaluate the performance and impacts of CAVs, and improve safety and efficiency for all road users. To minimize the safety risk, in parallel to the AV deployment efforts, techniques to test and evaluate (T&E)the AVs and their autonomy needed to be enhanced 103,104 . In addition to T&E efforts, fault detection 105,106 and fault accommodation 107,108 techniques needed to be developed to minimize the impact of failures of components of CAVs.…”

Section: Future Challenges For Development and Deployment Of Cavsmentioning

confidence: 99%

See 1 more Smart Citation

Connected autonomous vehicles: State of practice

Islam

Newaz

Song

et al. 2023

Appl Stoch Models Bus & Ind

Self Cite

View full text Add to dashboard Cite

Connected autonomous vehicles (CAVs) have the potential to deal with the steady increase in road traffic while solving transportation related issues such as traffic congestion, pollution, and road safety. Therefore, CAVs are becoming increasingly popular and viewed as the next generation transportation solution. Although modular advancements have been achieved in the development of CAVs, these efforts are not fully integrated to operationalize CAVs in realistic driving scenarios. This paper surveys a wide range of efforts reported in the literature about the CAV developments, summarizes the CAV impacts from a statistical perspective, explores current state of practice in the field of CAVs in terms of autonomy technologies, communication backbone, and computation needs. Furthermore, this paper provides general guidance on how transportation infrastructures need to be prepared in order to effectively operationalize CAVs. The paper also identifies challenges that need to be addressed in near future for effective and reliable adoption of CAVs.

show abstract

Section: Future Challenges For Development and Deployment Of Cavsmentioning

confidence: 99%

Section: Future Challenges For Development and Deployment Of Cavsmentioning

confidence: 99%

Connected autonomous vehicles: State of practice

Islam

Newaz

Song

et al. 2023

Appl Stoch Models Bus & Ind

Self Cite

View full text Add to dashboard Cite

show abstract

“…A previously proposed automatic simulation-based testing approach [15] combined domain expert knowledge and autonomous systems' operating and environmental parameters. During the testing procedure, type-2 fuzzy logic was applied to facilitate the robust management of data uncertainty.…”

Section: Related Literature Surveymentioning

confidence: 99%

Finite State GUI Testing with Test Case Prioritization Using Z-BES and GK-GRU

Kumar,

Nitin,

Yadav

2023

Applied Sciences

View full text Add to dashboard Cite

To deliver user-friendly experiences, modern software applications rely heavily on graphical user interfaces (GUIs). However, it is paramount to ensure the quality of these GUIs through effective testing. This paper proposes a novel “Finite state testing for GUI with test case prioritization using ZScore-Bald Eagle Search (Z-BES) and Gini Kernel-Gated recurrent unit (GK-GRU)” approach to enhance GUI testing accuracy and efficiency. First, historical project data is collected. Subsequently, by utilizing the Z-BES algorithm, test cases are prioritized, aiding in improving GUI testing. Attributes are then extracted from prioritized test cases, which contain crucial details. Additionally, a state transition diagram (STD) is generated to visualize system behavior. The state activity score (SAS) is then computed to quantify state importance using reinforcement learning (RL). Next, GUI components are identified, and their text values are extracted. Similarity scores between GUI text values and test case attributes are computed. Grounded on similarity scores and SAS, a fuzzy algorithm labels the test cases. Data representation is enhanced by word embedding using GS-BERT. Finally, the test case outcomes are predicted by the GK-GRU, validating the GUI performance. The proposed work attains 98% accuracy, precision, recall, f-measure, and sensitivity, and low FPR and FNR error rates of 14.2 and 7.5, demonstrating the reliability of the model. The proposed Z-BES requires only 5587 ms to prioritize the test cases, retaining less time complexity. Meanwhile, the GK-GRU technique requires 38945 ms to train the neurons, thus enhancing the computational efficiency of the system. In conclusion, experimental outcomes demonstrate that, compared with the prevailing approaches, the proposed technique attains superior performance.

show abstract

“…The simulation testing relies on virtual simulation environments to test perception algorithms. Although the accuracy of virtual simulation environments is increasing, virtual environments cannot completely replace real traffic scenes [5][6]. At the same time, the virtual testing scenario is fragmented, and autonomous vehicles are a continuous process during driving, the virtual testing does not meet the continuity requirements of autonomous driving scenarios [7].…”

Section: Introductionmentioning

confidence: 99%

Research on testing and evaluation method of autonomous driving perception algorithm based on data playback

Jin,

Wang,

Zhang

et al. 2024

International Conference on Mechatronic Engineering and Artificial Intelligence (MEAI 2023)

View full text Add to dashboard Cite

With the promotion of policies and the continuous development of artificial intelligence, autonomous vehicle have gained more and more attention. The environment perception system is the key to realize the real-time interaction between the vehicle and the external environment. It is also the first step to realize automatic driving, and plays an important role in the safe driving of autonomous vehicle. The development and application of the environmental perception algorithms cannot be separated from testing and evaluation. The test evaluation can effectively verify the accuracy and stability of the environment perception algorithm, and provide reliable input for the decision-making and control of the auto drive system. This article establishes a perception algorithm testing system based on data playback, and proposes a testing and evaluation method for perception algorithms based on the environmental information requirements of decision control systems, in order to improve the development efficiency of environmental perception algorithms.

show abstract

Automatic Test and Evaluation of Autonomous Systems

Cited by 7 publications

References 40 publications

Connected autonomous vehicles: State of practice

Connected autonomous vehicles: State of practice

Finite State GUI Testing with Test Case Prioritization Using Z-BES and GK-GRU

Research on testing and evaluation method of autonomous driving perception algorithm based on data playback

Contact Info

Product

Resources

About