Dynamic Upsampling of Smoke through Dictionary-based Learning

Abstract

Simulating turbulent smoke flows with fine details is computationally intensive. For iterative editing or simply faster generation, efficiently upsampling a low-resolution numerical simulation is an attractive alternative. We propose a novel learning approach to the dynamic upsampling of smoke flows based on a training set of flows at coarse and fine resolutions. Our multiscale neural network turns an input coarse animation into a sparse linear combination of small velocity patches present in a precomputed over-complete dictionary. These sparse coefficients are then used to generate a high-resolution smoke animation sequence by blending the fine counterparts of the coarse patches. Our network is initially trained from a sequence of example simulations to both construct the dictionary of corresponding coarse and fine patches and allow for the fast evaluation of a sparse patch encoding of any coarse input. The resulting network provides an accurate upsampling when the coarse input simulation is well approximated by patches present in the training set (e.g., for re-simulation), or simply visually plausible upsampling when input and training sets differ significantly. We show a variety of examples to ascertain the strengths and limitations of our approach and offer comparisons to existing approaches to demonstrate its quality and effectiveness.

Predicting high-resolution turbulence details in space and time

“…In CG for instance, they have been explored as a means to synthesize either super-resolution or upsampling of fluid flows. For super-resolution, the input velocity field is a downsampled version of a high-resolution simulation, and generative neural networks [Xie et al 2018;Werhahn et al 2019] or local dictionary-based neural network [Bai et al 2020] have been proposed to recover the missing fine details. For fluid upsampling, the input is a fluid simulation computed on a coarse grid, and methods to learn local patch descriptors [Chu and Thuerey 2017] or the formation of small-scale splashes [Um et al 2018] have been formulated to enrich a simulation.…”

“…In this paper, we introduce a simple but effective learning-based approach, which not only outperforms state-of-the-art methods for spatial upsampling of velocity fields at high Reynolds numbers, but also offers high-quality temporal upsampling of arbitrary fluid flows using the same network structure. Building upon the recent work of Bai et al [2020] which argued that fine spatial structures of turbulent flows are seemingly complex overall but locally simple, our method for predicting turbulent flow details also relies on a localized dictionary-based neural network that learns how to combine simple, local structures to obtain fine details globally. However, in a marked departure from [Bai et al 2020], we show that additional filtering in the process can dramatically improve the prediction quality of the neural network, and that further augmenting the input with time stamps also leads to high-quality temporal interpolation between frames.…”

“…Building upon the recent work of Bai et al [2020] which argued that fine spatial structures of turbulent flows are seemingly complex overall but locally simple, our method for predicting turbulent flow details also relies on a localized dictionary-based neural network that learns how to combine simple, local structures to obtain fine details globally. However, in a marked departure from [Bai et al 2020], we show that additional filtering in the process can dramatically improve the prediction quality of the neural network, and that further augmenting the input with time stamps also leads to high-quality temporal interpolation between frames. As a result, our novel localized dictionary-based neural network can efficiently synthesize turbulent flows both spatially and temporally from coarse inputs as shown in Fig.…”

Accelerated Smoke Simulation by Super-Resolution With Deep Learning on Downscaled and Binarized Space

“…Xie et al [28] and Werhahn et al [48] employed a GAN for both 2D and 3D SR smoke flow. Bai et al [47] used deep learning to perform SR that produced HR flow details with dynamic features.…”

Two‐step Temporal Interpolation Network Using Forward Advection for Efficient Smoke Simulation

“…A generative model applying a FLIP simulation has been proposed to improve the details of liquid splashing [UHT18], and generative models for super‐resolution have been developed to convert low‐resolution flow simulation results into high‐resolution results [XFCT18, WXCT19]. Bai et al [BLDL20] proposed a multiscale neural network that can upsample a coarse animation into a high‐resolution smoke animation via dictionary‐based learning. Moreover, DNN models that encode a flow simulation as a simplified representation and simulation methods using the simplified representation have also been advanced.…”