Repairing Learned Controllers with Convex Optimization: A Case Study

Guidotti, Dario; Leofante, Francesco; Castellini, Claudio; Tacchella, Armando

doi:10.1007/978-3-030-19212-9_24

Cited by 11 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If one has U < L, then the network is guaranteed to be robust to any small perturbations in L p norm. LP was also used by [89] in order to automatically reduce the intent detection mismatch of a prosthetic hand.…”

Section: Abstract Interpretationmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

Context: Machine Learning (ML) has been at the heart of many innovations over the past years. However, including it in so-called "safety-critical" systems such as automotive or aeronautic has proven to be very challenging, since the shift in paradigm that ML brings completely changes traditional certification approaches.Objective: This paper aims to elucidate challenges related to the certification of ML-based safety-critical systems, as well as the solutions that are proposed in the literature to tackle them, answering the question "How to Certify Machine Learning Based Safety-critical Systems?".Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems. In total, we identified 217 papers covering topics considered to be the main pillars of ML certification: Robustness, Uncertainty, Explainability, Verification, Safe Reinforcement Learning, and Direct Certification. We analyzed the main trends and problems of each sub-field and provided summaries of the papers extracted.Results: The SLR results highlighted the enthusiasm of the community for this subject, as well as the lack of diversity in term of datasets and type of ML models. It also emphasized the need to further develop connections between academia and industries to deepen the domain study. Finally, it also illustrated This work is supported by the DEEL project CRDPJ 537462-18 funded by the National Science and Engineering Research Council of Canada (NSERC) and the Consortium for Research and Innovation in Aerospace in Québec (CRIAQ), together with its industrial partners Thales Canada inc, Bell Textron Canada Limited, CAE inc and Bombardier inc.

show abstract

Section: Abstract Interpretationmentioning

confidence: 99%

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

Tambon,

Laberge,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Verification (Bak et al, 2020, Demarchi et al, 2022, Eramo et al, 2022, Ferrari et al, 2022, Guidotti, 2022, Guidotti et al, 2019b,b, 2020, 2023c,d,e, Henriksen and Lomuscio, 2021, Katz et al, 2019, Kouvaros et al, 2021, Singh et al, 2019a, which aims to provide formal assurances regarding the behavior of neural networks, has emerged as a potential solution to the aforementioned robustness issues. In addition to the development of verification tools and techniques, a substantial amount of research is also directed towards modifying networks to align with specified criteria (Guidotti et al, 2019a,b, Henriksen et al, 2022, Kouvaros et al, 2021, Sotoudeh and Thakur, 2021, and exploring methods for training networks that adhere to specific constraints on their behavior (Cohen et al, 2019, Eaton-Rosen et al, 2018, Giunchiglia and Lukasiewicz, 2021, Giunchiglia et al, 2022, Hu et al, 2016.…”

Section: Introductionmentioning

confidence: 99%

NeVer2: Learning and Verification of Neural Networks

Demarchi,

Guidotti,

Pulina

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

NeVer2 is a cross-platform open-source tool to design, train and verify neural networks. NeVer2 integrates off-the-shelf learning libraries with our verification backend and makes their functionalities available through a graphical interface. A user can work out the structure of a neural network by dragging and dropping blocks on a canvas and then train the network by supplying a dataset source and the hyper-parameters through dialog boxes. Once the network is trained, verification amounts to (i) adding input pre-conditions and output post-conditions to the network through special blocks and (ii) starting verification in a "push-button" fashion. To the best of our knowledge, there is no other publicly available tool that integrates all such features at present. Here we provide a detailed description of NeVer2 including examples to demonstrate the seamless integration between design, training and verification. We further provide an experimental analysis on several verification benchmarks in order to show the trade-off between completeness and computability using different algorithms, including a comparison with the state-of-the-art tools α,β-CROWN and NNV for reference.

show abstract

“…Providing formal guarantees on the performance of neural networks [6][7][8][9][10][11][12][13][14][15][16][17][18][19][20], known as verification, or making them compliant with such guarantees [21][22][23][24][25][26][27], known as repair, has proven to be similarly challenging, even when using models with limited complexity and size. Additionally, neural networks have recently been found to be prone to reliability issues known as adversarial perturbations [28], where seemingly insignificant variations in their inputs cause unforeseeable and undesirable changes in their behavior.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Satisfiability Modulo Theory Solvers for Verification of Neural Networks in Predictive Maintenance Applications

2023

View full text Add to dashboard Cite

Interest in machine learning and neural networks has increased significantly in recent years. However, their applications are limited in safety-critical domains due to the lack of formal guarantees on their reliability and behavior. This paper shows recent advances in satisfiability modulo theory solvers used in the context of the verification of neural networks with piece-wise linear and transcendental activation functions. An experimental analysis is conducted using neural networks trained on a real-world predictive maintenance dataset. This study contributes to the research on enhancing the safety and reliability of neural networks through formal verification, enabling their deployment in safety-critical domains.

show abstract

Repairing Learned Controllers with Convex Optimization: A Case Study

Cited by 11 publications

References 24 publications

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

NeVer2: Learning and Verification of Neural Networks

Leveraging Satisfiability Modulo Theory Solvers for Verification of Neural Networks in Predictive Maintenance Applications

Contact Info

Product

Resources

About