Data-driven profile prediction for DIII-D

Abbate, Joseph; Conlin, Rory; Kolemen, Egemen

doi:10.1088/1741-4326/abe08d

Cited by 21 publications

(28 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These combined benefits reduce the controller development cycle and accelerate the study of alternative plasma configurations. Indeed, artificial intelligence has recently been identified as a ‘Priority Research Opportunity’ for fusion control 14 , building on demonstrated successes in reconstructing plasma-shape parameters 15 , 16 , accelerating simulations using surrogate models 17 , 18 and detecting impending plasma disruptions 19 . RL has not, however, been used for magnetic controller design, which is challenging due to high-dimensional measurements and actuation, long time horizons, rapid instability growth rates and the need to infer the plasma shape through indirect measurements.…”

Section: Mainmentioning

confidence: 99%

Magnetic control of tokamak plasmas through deep reinforcement learning

Degrave

Felici

Buchli

et al. 2022

Nature

454

218

View full text Add to dashboard Cite

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.

show abstract

Section: Mainmentioning

confidence: 99%

Magnetic control of tokamak plasmas through deep reinforcement learning

Degrave

Felici

Buchli

et al. 2022

Nature

454

218

View full text Add to dashboard Cite

show abstract

“…2020; Yellapantula et al. 2020; Abbate, Conlin & Kolemen 2021; Bai & Peng 2021; Hatfield et al. 2021; Maschler & Weyrich 2021; Brunton & Kutz 2022).…”

Section: Background and Previous Workmentioning

confidence: 99%

Towards fast and accurate predictions of radio frequency power deposition and current profile via data-driven modelling: applications to lower hybrid current drive

et al. 2022

View full text Add to dashboard Cite

Three machine learning techniques (multilayer perceptron, random forest and Gaussian process) provide fast surrogate models for lower hybrid current drive (LHCD) simulations. A single GENRAY/CQL3D simulation without radial diffusion of fast electrons requires several minutes of wall-clock time to complete, which is acceptable for many purposes, but too slow for integrated modelling and real-time control applications. The machine learning models use a database of more than 16 000 GENRAY/CQL3D simulations for training, validation and testing. Latin hypercube sampling methods ensure that the database covers the range of nine input parameters ( $n_{e0}$ , $T_{e0}$ , $I_p$ , $B_t$ , $R_0$ , $n_{\|}$ , $Z_{{\rm eff}}$ , $V_{{\rm loop}}$ and $P_{{\rm LHCD}}$ ) with sufficient density in all regions of parameter space. The surrogate models reduce the inference time from minutes to $\sim$ ms with high accuracy across the input parameter space.

show abstract

“…Control Problems. We tackle five control problems: the standard underactuated pendulum swing-up problem (Pendulum-v0 from Brockman et al [2016]), a cartpole swing-up problem, a 2D lava path navigation problem, a 2-DOF robot arm reacher problem with 8-dimensional state (Reacher-v2 from Brockman et al [2016]), and a simplified beta tracking problem from plasma control [Char et al, 2019, Mehta et al, 2020 where the controller must maintain a fixed normalized plasma pressure using as GT dynamics a model learned similarly to Abbate et al [2021]. The lava path is intended to test stability and exploration of algorithms.…”

Section: Environmentmentioning

confidence: 99%

“…The action is the next change in the power injection level. The "ground-truth" dynamics for this problem are given by a neural network model learned from data processed as in Abbate et al [2021]. Control is done at a timestep of 200ms and the reward function is the negative absolute deviation from β n = 2.…”

Section: Description Of Continuous Control Problemsmentioning

confidence: 99%

An Experimental Design Perspective on Model-Based Reinforcement Learning

Mehta¹,

Paria²,

Schneider³

et al. 2021

Preprint

View full text Add to dashboard Cite

In many practical applications of RL, it is expensive to observe state transitions from the environment. For example, in the problem of plasma control for nuclear fusion, computing the next state for a given state-action pair requires querying an expensive transition function which can lead to many hours of computer simulation or dollars of scientific research. Such expensive data collection prohibits application of standard RL algorithms which usually require a large number of observations to learn. In this work, we address the problem of efficiently learning a policy while making a minimal number of state-action queries to the transition function. In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning. We propose an acquisition function that quantifies how much information a state-action pair would provide about the optimal solution to a Markov decision process. At each iteration, our algorithm maximizes this acquisition function, to choose the most informative state-action pair to be queried, thus yielding a data-efficient RL approach. We experiment with a variety of simulated continuous control problems and show that our approach learns an optimal policy with up to 5 -1, 000× less data than model-based RL baselines and 10 3 -10 5 × less data than model-free RL baselines. We also provide several ablated comparisons which point to substantial improvements arising from the principled method of obtaining data.

show abstract

Data-driven profile prediction for DIII-D

Cited by 21 publications

References 44 publications

Magnetic control of tokamak plasmas through deep reinforcement learning

Magnetic control of tokamak plasmas through deep reinforcement learning

Towards fast and accurate predictions of radio frequency power deposition and current profile via data-driven modelling: applications to lower hybrid current drive

An Experimental Design Perspective on Model-Based Reinforcement Learning

Contact Info

Product

Resources

About