Extremum seeking for optimal control problems with unknown time‐varying systems and unknown objective functions

Scheinker, Alexander; Scheinker, David

doi:10.1002/acs.3097

Cited by 13 publications

(11 citation statements)

References 53 publications

(96 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our case we prefer to use a recently developed robust extremum seeking (ES) feedback control algorithm which was originally designed for the stabilization of unknown open-loop unstable time-varying nonlinear systems and then extended for the optimization of many parameter time-varying unknown systems with noise-corrupted measurement functions [67,68]. This method can quickly tune many coupled parameters and has been applied for time-varying problems such as adaptively learning optimal feedback control policies for unknown time-varying systems directly from measurement data [69].…”

Section: Overview Of Main Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Latent Space Tuning for Non-Stationary Distributions

Scheinker¹,

Cropp²,

Paiagua³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Powerful deep learning tools, such as convolutional neural networks (CNN), are able to learn the input-output relationships of large complicated systems directly from data. Encoder-decoder deep CNNs are able to extract features directly from images, mix them with scalar inputs within a general low-dimensional latent space, and then generate new complex 2D outputs which represent complex physical phenomenon. One important challenge faced by deep learning methods is large non-stationary systems whose characteristics change quickly with time for which re-training is not feasible. In this paper we present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs based on real-time feedback to quickly compensate for unknown and fast distribution shifts. We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator whose components (accelerating electric fields and focusing magnetic fields) are also quickly changing with time.

show abstract

Section: Overview Of Main Resultsmentioning

confidence: 99%

“…The adaptive feedback dynamics (13) are chosen based on the results in [67][68][69]. The hyperparameters in (13) can intuitively be understood as α representing a dithering amplitude which controls the size of the dynamic perturbations, k a feedback gain, and the product kα can be thought of as an overall learning rate.…”

Section: Adaptive Latent Space Tuningmentioning

confidence: 99%

Adaptive Latent Space Tuning for Non-Stationary Distributions

Scheinker¹,

Cropp²,

Paiagua³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reinforcement learning has recently grown in popularity with the use of ML methods being utilized to learn models for Dynamic Programming problems in order to satisfy the Bellman optimality condition [14]. In [15], a reinforcement learning approach is utilized in which optimal feedback control laws (or "agent policies") are learned online directly from system data for unknown and timevarying systems. This approach was studied for the optimal control of radio frequency accelerating cavities with characteristics that drift with time, such as cable length changes, resonance frequencies, and analog component fluctuations, due to temperature variations.…”

Section: Ml-enhanced Controlmentioning

confidence: 99%

Virtual Diagnostic Suite for Electron Beam Prediction and Control at FACET-II

et al. 2021

Self Cite

View full text Add to dashboard Cite

We discuss the implementation of a suite of virtual diagnostics at the FACET-II facility currently under commissioning at SLAC National Accelerator Laboratory. The diagnostics will be used for the prediction of the longitudinal phase space along the linac, spectral reconstruction of the bunch profile, and non-destructive inference of transverse beam quality (emittance) while using edge radiation at the injector dogleg and bunch compressor locations. These measurements will be folded into adaptive feedbacks and Machine Learning (ML)-based reinforcement learning controls to improve the stability and optimize the performance of the machine for different experimental configurations. In this paper we describe each of these diagnostics with expected measurement results that are based on simulation data and discuss progress towards implementation in regular operations.

show abstract

“…where f and g are both nonlinear, time-varying, and analytically unknown. The method has now been generalized further with analytical proofs of stability for non-differentiable systems as well as systems not affine in control [41,42], of the forṁ x = f(x, u, t), and has been utilized in various particle accelerator applications [26,[29][30][31][32]43].…”

Section: Unknown Time-varying Systems and Adaptive Feedback Controlmentioning

confidence: 99%

“…Recently, methods such as reinforcement learning have been developed for this optimal control problem for unknown systems with unknown dynamics f and measurable, but analytically unknown cost function C for which it is impossible to calculate the optimal control policy with the standard Dynamic Programming approach [4]. One example is to use adaptive methods which can be applied for online RL in which optimal feedback control policies are learned directly from data to learn optimal feedback control policies which are parametrized by a chosen set of basis functions whose coefficients must be adaptively tuned online [43]. Other RL approaches learn either the unknown dynamics f , or the cost function C, or both directly from measured data and represent them as trained neural networks, as described above.…”

Section: Machine Learning For Time-varying Systemsmentioning

confidence: 99%

Adaptive Machine Learning for Robust Diagnostics and Control of Time-Varying Particle Accelerator Components and Beams

Scheinker

2021

Information

Self Cite

View full text Add to dashboard Cite

Machine learning (ML) is growing in popularity for various particle accelerator applications including anomaly detection such as faulty beam position monitor or RF fault identification, for non-invasive diagnostics, and for creating surrogate models. ML methods such as neural networks (NN) are useful because they can learn input-output relationships in large complex systems based on large data sets. Once they are trained, methods such as NNs give instant predictions of complex phenomenon, which makes their use as surrogate models especially appealing for speeding up large parameter space searches which otherwise require computationally expensive simulations. However, quickly time varying systems are challenging for ML-based approaches because the actual system dynamics quickly drifts away from the description provided by any fixed data set, degrading the predictive power of any ML method, and limits their applicability for real time feedback control of quickly time-varying accelerator components and beams. In contrast to ML methods, adaptive model-independent feedback algorithms are by design robust to un-modeled changes and disturbances in dynamic systems, but are usually local in nature and susceptible to local extrema. In this work, we propose that the combination of adaptive feedback and machine learning, adaptive machine learning (AML), is a way to combine the global feature learning power of ML methods such as deep neural networks with the robustness of model-independent control. We present an overview of several ML and adaptive control methods, their strengths and limitations, and an overview of AML approaches.

show abstract

Extremum seeking for optimal control problems with unknown time‐varying systems and unknown objective functions

Cited by 13 publications

References 53 publications

Adaptive Latent Space Tuning for Non-Stationary Distributions

Adaptive Latent Space Tuning for Non-Stationary Distributions

Virtual Diagnostic Suite for Electron Beam Prediction and Control at FACET-II

Adaptive Machine Learning for Robust Diagnostics and Control of Time-Varying Particle Accelerator Components and Beams

Contact Info

Product

Resources

About