Adaptive Deep Learning for Time-Varying Systems With Hidden Parameters: Predicting Changing Input Beam Distributions of Compact Particle Accelerators

Abstract

Machine learning (ML) tools are able to learn relationships between the inputs and outputs of large complex systems directly from data. However for time-varying systems, the predictive capabilities of ML tools degrade if the systems are no longer accurately represented by the data with which the ML models were trained. For complex systems, re-training is only possible if the changes are slow relative to the rate at which large numbers of new input-output training data can be non-invasively recorded. In this work, we present an approach to deep learning for time-varying systems which does not require re-training. Our approach is to include adaptive feedback in the architecture of deep generative convolutional neural networks (CNN). The feedback is based only on available system output measurements and is applied in the encoded low-dimensional dense layers of the encoder-decoder CNNs. Our approach is inspired by biological systems in which separate groups of neurons interact and are controlled and synchronized by external feedbacks. We demonstrate this approach by developing an inverse model of a complex charged particle accelerator system, mapping output beam measurements to input beam distributions, while both the accelerator components and the unknown input beam distribution vary rapidly with time. We demonstrate our methods on experimental measurements of the input and output beam distributions of the HiRES ultra-fast electron diffraction (UED) beam line at Lawrence Berkeley National Laboratory. Our method can be successfully used to aid both physics and ML-based surrogate online models to provide non-invasive beam diagnostics. We also demonstrate our method for automatically tracking the time varying quantum efficiency map of a particle accelerator’s photocathode.

Adaptive machine learning for time-varying systems: low dimensional latent space tuning

“…Adaptive latent space tuning maps high dimension inputs down to extremely low dimensional latent space representations of generative encoder-decoder style convolutional neural networks (CNN) [29,30]. Encoder-decoder CNNs are powerful ML tools that can represent complex relationships in high-dimensional systems in a low dimensional latent space [31,32], and have been used for anomaly detection [33], for time-series data [34], for optimization of deep generative models [35].…”

“…A new adaptive machine learning (AML) approach for time varying systems has recently been proposed in which model-independent adaptive feedback is used to tune the low dimensional latent space of encoder-decoder architecture convolutional neural networks (CNN) for systems that change with time [29,30]. The encoder half of the CNN can use both images (initial beam distributions) and scalars (accelerator magnet and RF parameter settings) as inputs with the scalars being concated together with the information from the images once the images have been scaled down using 2D convolutional layers, flattened, and passed through dense fully connected layers.…”

2022 Review of Data-Driven Plasma Science

“…Novel AML methods are being developed which utilize adaptive feedback to tune the low dimensional latent space of encoder-decoder type convolutional neural networks based on real-time measurements and for online adjustment of inverse models that can provide realistic estimate of the accelerator's input beam's phase space distribution based only on downstream diagnostics [287,288]. Such AML tools have the potential to enable truly autonomous accelerator controls and diagnostics so that they can continuously respond to un-modeled changes and disturbances in real time and thereby keep the accelerator performance (beam energy and energy spread, beam loss, phase space quality, etc) at a global optimal, not allowing it to drift as things change with time.…”

Simulations of future particle accelerators: issues and mitigations

“…Recently a LiTrack model of FACET was adaptively tuned online to match the simulated beam's energy spread spectrum ρ𝐸 (𝑥, 𝑡) to its measured 𝜌 𝐸 (𝑥, 𝑡), minimizing the error 𝑒(𝑡) = ∫ | ρ𝐸 (𝑥, 𝑡) − 𝜌 𝐸 (𝑥, 𝑡)| 𝑑𝑥, to track the time-varying longitudinal phase space (𝑧, 𝐸) of the electron beam [81,82]. Efforts are also underway to utilize ML tools to map diagnostics back to input beam distributions to be used as the initial conditions of accelerator models [83]. Such approaches are possible with any simulation tool for tracking time-varying beams.…”