Exploratory Lagrangian-Based Particle Tracing Using Deep Learning

Abstract

Time-varying vector fields produced by computational fluid dynamics simulations are often prohibitively large and pose challenges for accurate interactive analysis and exploration. To address these challenges, reduced Lagrangian representations have been increasingly researched as a means to improve scientific time-varying vector field exploration capabilities. This paper presents a novel deep neural network-based particle tracing method to explore time-varying vector fields represented by Lagrangian flow maps. In our workflow, in situ processing is first utilized to extract Lagrangian flow maps, and deep neural networks then use the extracted data to learn flow field behavior. Using a trained model to predict new particle trajectories offers a fixed small memory footprint and fast inference. To demonstrate and evaluate the proposed method, we perform an in-depth study of performance using a well-known analytical data set, the Double Gyre. Our study considers two flow map extraction strategies as well as the impact of the number of training samples and integration durations on efficacy, evaluates multiple sampling options for training and testing, and informs hyperparameter settings. Overall, we find our method requires a fixed memory footprint of 10.5 MB to encode a Lagrangian representation of a time-varying vector field while maintaining accuracy. For post hoc analysis, loading the trained model costs only two seconds, significantly reducing the burden of I/O when reading data for visualization. Moreover, our parallel implementation can infer one hundred locations for each of two thousand new pathlines across the entire temporal resolution in 1.3 seconds using one NVIDIA Titan RTX GPU.

DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization

“…This makes unsupervised learning, particularly self-supervised learning, a suitable candidate for accomplishing such tasks. Thus, investigating the underexplored self-supervised learning solutions for making predictions or recommendations will certainly boost [2], [27], [45], [46], [47], [49], [50] [12], [63] [35], [57] [11], [15] [24] + [51], [53], [54], [55] , [56], [76], [84] [70], [158] [71], [85] [31], [40] [126], [131], [156] + , [160], [162] [159]…”

“…We also refer to their research tasks when categorizing the surveyed papers in the respective tables according to learning type, network architecture, loss function, and evaluation metric. The description [51] TSR-TVD TVCG Han and Wang [50] SSR-TVD TVCG Han et al [55] STNet TVCG Wurster et al [167] arXiv Guo et al [47] SSR-VFD PVIS Jakob et al [76] TVCG Sahoo and Berger [126] IA-VFS EVIS An et al [2] STSRNet CG&A Han and Wang [53] TSR-VFD C&G Xie et al [168] tempoGAN TOG Werhahn et al [162] CGIT Wang et al [156] DeepOrganNet TVCG Lu et al [109] neurcomp CGF Weiss et al [160] fV-SRN arXiv Shi et al [131] GNN-Surrogate TVCG Han and Wang [54] VCNet VI Liu et al [106] JOV Han et al [49] CG&A Gu et al [45] VFR-UFD CG&A Han et al [56] V2V TVCG Gu et al [46] Scalar2Vec PVIS Kim et al [84] Deep Fluids CGF Chu et al [27] TOG Wiewel et al [163] LSP CGF Wiewel et al [164] LSS CGF Berger et al [12] TVCG Hong et al [70] DNN-VolVis PVIS He et al [63] InSituNet TVCG Weiss et al [159] TVCG Weiss et al [161] TVCG Weiss and Navab [158] DeepDVR arXiv He et al [62] CECAV-DNN VI Tkachev et al [143] TVCG Hong et al [71] PVIS Kim and Günther [85] CGF Han et al [57] arXiv Yang et al [169] JOV Shi and Tao [130] TIST Engel and Ropinski …”

“…LSP can achieve 150× speedups compared with a regular pressure solver, a significant boost in simulation performance. Wiewel et al [164] proposed latent space [12] new viewpoint and transfer function synthesized rendering conditioned on input Hong et al [70] DNN-VolVis original rendering, goal effect, new viewpoint synthesized rendering conditioned on input He et al [63] InSituNet ensemble simulation parameters synthesized rendering conditioned on input Weiss et al [159] low-resolution isosurface maps, optical flow high-resolution isosurface maps Weiss et al [161] low-resolution image high-resolution image Weiss and Navab [158] DeepDVR volume, viewpoint rendering image CECAV-DNN sequence of ensemble pairs likelihood each member from one ensemble Tkachev et al [143] local spatiotemporal patch future voxel value at patch center Hong et al [71] movement sequence probability vector of next movement Kim and Günther [85] unsteady 2D vector field reference frame transformation Han et al [57] particle start location, file cycles particle end location Yang et al [169] volume rendering under viewpoint viewpoint quality score Shi and Tao [130] volume rendering image estimated viewpoint Engel and Ropinski [35] DVAO intensity volume, opacity volume or transfer function AO volume subdivision (LSS), an end-to-end DL-solution for robust prediction future timesteps of complex fluid simulations with high temporal stability. Using CNN and stacked LSTM, LSS achieves both spatial compression and temporal prediction.…”

“…Their solution can tackle noisy inputs and data with resampling artifacts by performing filtering and reference frame extraction end-to-end. Han et al [57] predicted particle end locations for Lagrangianbased particle tracing. Their MLP-based model learns particle end locations given their start locations and file cycles.…”

“…A close look at the categorization under CNN shows that encoder+decoder is the most diverse, covering all five research tasks, while decoder covers two tasks (i.e., data [2](U-Net, deformable CNN), [47] [11](CNN vs. ResNet vs. U-Net), [15](U-Net), [27] (GAN) [49] [50](GAN), [55] (GAN) [35], [40](nested encoder+decoder), [45], [46], [53] [76](ESPCN), [106] (GAN), [126] [54] (GAN), [56] (GAN), [70] (GAN) [159] (FRVSR-Net), [161](EnhanceNet) [81](symmetric FCN), [84](AE), [103](residual AE) [162] (multi-pass GAN) [116](U-Net vs. U-Net+ResNet vs. DenseNet) [167] (ESRGAN, WGAN) [117](AE), [154](multi-stream CNN) [168] (GAN), [182] [158](U-Net, V-Net), [163], [164] MLP [57], [183](FCCNN) - [84], [109], [160] RNN [164](stacked LSTM) [163](LSTM) -GNN [52](GCN), [64](GCN) [131](GCN) -CNN+MLP [24], [26](Siamese) [63] [12] (GAN), [48](AE) [31], [62], [85] [97](Geo-CNN)…”