Model Assertions for Monitoring and Improving ML Models

Kang, Daniel; Raghavan, Deepti; Bailis, Peter; Zaharia, Matei

doi:10.48550/arxiv.2003.01668

Cited by 4 publications

(6 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Aside from multi-view disagreements, our framework can be easily extended to any type of assertion such as class entropy, high network loss, etc. While there exists literature about model assertions [12], we are novel in which we apply the checks during training and inference of an existing dataset while theirs does it for Active Learning.…”

Section: Assertion-guided Point Samplingmentioning

confidence: 99%

AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation

Liong¹,

Nguyen²,

Widjaja³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper, we present an Assertion-based Multi-View Fusion network (AMVNet) for LiDAR semantic segmentation which aggregates the semantic features of individual projection-based networks using late fusion. Given class scores from different projection-based networks, we perform assertion-guided point sampling on score disagreements and pass a set of point-level features for each sampled point to a simple point head which refines the predictions. This modular-and-hierarchical late fusion approach provides the flexibility of having two independent networks with a minor overhead from a light-weight network. Such approaches are desirable for robotic systems, e.g. autonomous vehicles, for which the computational and memory resources are often limited. Extensive experiments show that AMVNet achieves state-of-the-art results in both the SemanticKITTI and nuScenes benchmark datasets and that our approach outperforms the baseline method of combining the class scores of the projection-based networks.

show abstract

Section: Assertion-guided Point Samplingmentioning

confidence: 99%

AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation

Liong¹,

Nguyen²,

Widjaja³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Dutta et al [24] present FLEX which uses approximate assertions to compare the actual and expected values, while systematically identify the acceptable bound between the actual and expected output which minimizes flakiness. Kang et al [36] introduce model assertions -that could be 'exact' or 'soft', which adapts the classical use of program assertions as a way to monitor and improve ML models.…”

Section: Specific Applicationsmentioning

confidence: 99%

“…Corduroy [54] introduces metamorphic assertions, built on top of Java Modelling Language (JML). Model assertions [36] adapt the classical use of program assertions, tailored to the specific needs of ML programs, in particular uncertainty in output.…”

Section: Techniquementioning

confidence: 99%

A Literature Survey of Assertions in Software Testing

Taromirad,

Runeson

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Assertions are one of the most useful automated techniques for checking program's behaviour and hence have been used for different verification and validation tasks. We provide an overview of the last two decades of research involving 'assertions' in software testing. Based on a term-based search, we filtered the inclusion of relevant papers and synthesised them w.r.t. the problem addressed, the solution designed, and the evaluation conducted. The survey rendered 119 papers on assertions in software testing. After test oracle, the dominant problem focus is test generation, followed by engineering aspects of assertions. Solutions are typically embedded in tool prototypes and evaluated throughout limited number of cases while using large-scale industrial settings is still a noticeable method. We conclude that assertions would be worth more attention in future research, particularly regarding the new and emerging demands (e.g., verification of programs with uncertainty), for effective, applicable, and domain-specific solutions.

show abstract

“…Comparing the model's reasoning process to human expectations is a powerful tool for finding bugs, but less broadly applicable than approaches that only require level 2 access. Approaches for debugging models include model assertions [41], editing models, regularization, data augmentation, data pre-processing, prediction post-processing, and anomaly detection [34]. Among these methods, the vast majority are compatible with level 2 access.…”

Section: The Utility Of Interpretability Beyond Trustmentioning

confidence: 99%

Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

Shen¹

2022

Preprint

View full text Add to dashboard Cite

The problem of human trust in artificial intelligence is one of the most fundamental problems in applied machine learning. Our processes for evaluating AI trustworthiness have substantial ramifications for ML's impact on science, health, and humanity, yet confusion surrounds foundational concepts. What does it mean to trust an AI, and how do humans assess AI trustworthiness? What are the mechanisms for building trustworthy AI? And what is the role of interpretable ML in trust?Here, we draw from statistical learning theory and sociological lenses on human-automation trust to motivate an AI-as-tool framework, which distinguishes human-AI trust from human-AI-human trust. Evaluating an AI's contractual trustworthiness involves predicting future model behavior using behavior certificates (BCs) that aggregate behavioral evidence from diverse sources including empirical out-of-distribution and out-of-task evaluation and theoretical proofs linking model architecture to behavior.We clarify the role of interpretability in trust with a ladder of model access. Interpretability (level 3) is not necessary or even sufficient for trust, while the ability to run a black-box model at-will (level 2) is necessary and sufficient. While interpretability can offer benefits for trust, it can also incur costs. We clarify ways interpretability can contribute to trust, while questioning the perceived centrality of interpretability to trust in popular discourse.How can we empower people with tools to evaluate trust? Instead of trying to understand how a model works, we argue for understanding how a model behaves. Instead of opening up black boxes, we should create more behavior certificates that are more correct, relevant, and understandable. We discuss how to build trusted and trustworthy AI responsibly with contract-aware model design, robustness testing, and favoring trust calibration over maximization. CCS Concepts: • Computing methodologies → Artificial intelligence; • Social and professional topics → Computing / technology policy; • Human-centered computing → Interaction design theory, concepts and paradigms.

show abstract

Model Assertions for Monitoring and Improving ML Models

Cited by 4 publications

References 32 publications

AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation

AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation

A Literature Survey of Assertions in Software Testing

Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

Contact Info

Product

Resources

About