Learning Convex Optimization Models

Agrawal, Akshay; Barratt, Shane; Boyd, Stephen

doi:10.48550/arxiv.2006.04248

Cited by 2 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Toy Experiment: In this experiment, the VAE Encoder is made up of 3 CNN layers with filter sizes of [ (3,3), (4,4), (5,5)], strides of [1, 2, 2], and padding of [1, 1, 2] respectively. Similarly, the decoder is made up of 3 CNN layers with filter sizes of [ (6,6), (6,6), (5,5)], strides of [2, 2, 1], and padding of [2,2,2] respectively.…”

Section: Appendixmentioning

confidence: 99%

“…In stark contrast, we focus on fixed, model-based control policies (often represented by a convex program), and instead focus on re-training perception models that provide state estimates that serve as input parameters for convex MPC. To do so, we leverage recent advances in differentiable MPC [5], [4], [7] to compute the sensitivity of the control cost to erroneous perceptual inputs. As such, we leverage the structure of the model-based controller to efficiently synthesize adversarial inputs as opposed to model-free RARL methods.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Control

Agarwal¹,

Chinchali²

2022

Preprint

View full text Add to dashboard Cite

Today's robots often interface data-driven perception and planning models with classical model-based controllers. For example, drones often use computer vision models to estimate navigation waypoints that are tracked by model predictive control (MPC). Often, such learned perception/planning models produce erroneous waypoint predictions on out-ofdistribution (OoD) or even adversarial visual inputs, which increase control cost. However, today's methods to train robust perception models are largely task-agnostic -they augment a dataset using random image transformations or adversarial examples targeted at the vision model in isolation. As such, they often introduce pixel perturbations that are ultimately benign for control, while missing those that are most adversarial. In contrast to prior work that synthesizes adversarial examples for single-step vision tasks, our key contribution is to efficiently synthesize adversarial scenarios for multi-step, model-based control. To do so, we leverage differentiable MPC methods to calculate the sensitivity of a model-based controller to errors in state estimation, which in turn guides how we synthesize adversarial inputs. We show that re-training vision models on these adversarial datasets improves control performance on OoD test scenarios by up to 28.2% compared to standard taskagnostic data augmentation. Our system is tested on examples of robotic navigation and vision-based control of an autonomous air vehicle.

show abstract

Section: Appendixmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Control

Agarwal¹,

Chinchali²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…KKT conditions) the implicit function theorem can be used to compute gradients. This was done for quadratic programs in [2], embedding MaxSAT in neural networks [62], a large class of convex optimization problems [1], smoothed top-k selection via optimal transport [67] and deep equilibrium models [4].…”

Section: End-to-end Trainingmentioning

confidence: 99%

Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach

Abbas¹,

Swoboda²

2021

Preprint

View full text Add to dashboard Cite

We propose an end-to-end trainable architecture for simultaneous semantic and instance segmentation (a.k.a. panoptic segmentation) consisting of a convolutional neural network and an asymmetric multiway cut problem solver. The latter solves a combinatorial optimization problem that elegantly incorporates semantic and boundary predictions to produce a panoptic labeling. Our formulation allows to directly maximize a smooth surrogate of the panoptic quality metric by backpropagating the gradient through the optimization problem. Experimental evaluation shows improvement of end-to-end learning w.r.t. comparable approaches on Cityscapes and COCO datasets. Overall, our approach shows the utility of using combinatorial optimization in tandem with deep learning in a challenging large scale real-world problem and showcases benefits and insights into training such an architecture end-to-end.Preprint. Under review.

show abstract

Learning Convex Optimization Models

Cited by 2 publications

References 29 publications

Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Control

Synthesizing Adversarial Visual Scenarios for Model-Based Robotic Control

Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach

Contact Info

Product

Resources

About