An interactive scientific simulation and visualization pipeline in mixed reality was developed to explore how scientific simulation could proceed in coming workplace. The main technical concern is fast scientific simulator and visualizer in mixed reality with hand gesture. Framework for scientific simulation and visualization was implemented on the server side based on deep learning based fast simulation model and open source visualization toolkit, VTK, respectively. Airfoil aerodynamics was selected as a target scientific simulation task since it is a representative and critical problem in scientific fluid simulation determining the whole aero-product's performance and shows highly nonlinear physics even by small shape deviation and so high computational cost. Therefore, the fast simulator was firstly developed by carefully developing convolutional neural network-based model, using U-net, which learned corresponding high accuracy simulation dataset. All scenes which are supposed to be seen on the mobile device side were authored in a widely used content development tool called Unity3D with a mixed reality toolkit, MRTK for Microsoft Hololens 2. User interface and user experience were created based on a scenario of a drone rotor which demonstrates running and analyzing the simulation results using designed interaction gestures. Finally, several discussions were derived through the implemented digital twin pipeline. It was concluded, with an adequately implemented mixed reality tools, that scientific researchers or industrial engineers will be helped not only in individual analysis works due to the delicacy of hands and fingers and real time interaction in mixed reality, but also, as a future study, in cooperative analysis works.
INDEX TERMSComputational fluid dynamics, Computer simulation, Convolutional neural network, Digital twin, Extended reality, Mixed realityTABLE I NOMENCLATURE Symbols or Abbreviations Meanings ρ Density p Pressure t, x, y Time and spatial axis u, v Velocity on x, y-axis 2D or 3D Two-dimensional or three-dimensional AI Artificial intelligence AR Augmented reality CFD Computational fluid dynamics CNN Convolution neural network ELU Exponential linear unit I/O Input and output LiDAR Light detection and ranging MR Mixed reality MRO Maintenance, repair, and operation MRTK Mixed reality toolkit PDE Partial differential equation This article has been accepted for publication in IEEE Access.