NNVA: Neural Network Assisted Visual Analysis of Yeast Cell Polarization Simulation

Abstract

Fig. 1: Approach Overview: A trained neural network-based surrogate model acts as the backend analysis framework, driving our interactive visual analysis system for analyzing a computationally expensive yeast simulation model. Abstract-Complex computational models are often designed to simulate real-world physical phenomena in many scientific disciplines. However, these simulation models tend to be computationally very expensive and involve a large number of simulation input parameters, which need to be analyzed and properly calibrated before the models can be applied for real scientific studies. We propose a visual analysis system to facilitate interactive exploratory analysis of high-dimensional input parameter space for a complex yeast cell polarization simulation. The proposed system can assist the computational biologists, who designed the simulation model, to visually calibrate the input parameters by modifying the parameter values and immediately visualizing the predicted simulation outcome without having the need to run the original expensive simulation for every instance. Our proposed visual analysis system is driven by a trained neural network-based surrogate model as the backend analysis framework. In this work, we demonstrate the advantage of using neural networks as surrogate models for visual analysis by incorporating some of the recent advances in the field of uncertainty quantification, interpretability and explainability of neural network-based models. We utilize the trained network to perform interactive parameter sensitivity analysis of the original simulation as well as recommend optimal parameter configurations using the activation maximization framework of neural networks. We also facilitate detail analysis of the trained network to extract useful insights about the simulation model, learned by the network, during the training process. We performed two case studies, and discovered multiple new parameter configurations, which can trigger high cell polarization results in the original simulation model. We evaluated our results by comparing with the original simulation model outcomes as well as the findings from previous parameter analysis performed by our experts.

GNN-Surrogate: A Hierarchical and Adaptive Graph Neural Network for Parameter Space Exploration of Unstructured-Mesh Ocean Simulations

“…Its major limitation is that the simulation output is only visualized with several predefined visual mappings, meaning scientists cannot adjust visual mappings to find features of interest after models have been trained. Second, researchers have also used different techniques such as machine learning [4], [5] and Gaussian process [21], [22] to predict raw data using surrogate models. Hazarika et al [4] trained a surrogate model to approximate the yeast cell polarization simulation model in the NNVA system.…”

“…To make the parameter space exploration efficient, scientists utilize and train a surrogate model by sampling parameter set-tings from the parameter space. Existing surrogate model-based parameter space analyses usually are either image-based (e.g., InSituNet [3]) or focus on regular grids [4], [5], which leads to two limitations. First, image-based surrogate models visualize the generated simulation data with several predefined visual mappings so that scientists are not able to adjust the setting of visual mappings to find features of interest after the models have been trained.…”

The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

“…During development , they improved their tool and received user feedback . Hazarika et al [HLW*20] used networks as surrogate models for visual analysis, and after the development of their system and techniques, a domain expert gave them feedback in order to further improve the VA system at the end of the development process. Ultimately, from the further analysis of the statistics, we conclude that in five cases both domain and ML experts used visualization tools and evaluated them.…”

“… Topic 10 – neurons’ activations. When visualizing DL techniques, the existing research has tried to address the activation of neurons in NN and their visual representation [HDK*19, HLW*20, HPRC20]. Different visualization techniques (e.g., 2D saliency/activation maps) have been used to visualize the activations of such neurons in various DL models, especially for image applications [AJY*18].…”

A novel no-sensors 3D model reconstruction from monocular video frames for a dynamic environment

“…In the past few years, there was a great interest to solve 3D scene reconstruction with moving objects using single or multiple Monocular camera RGB frames. Xingbin et al ( Yang, 2020 ), presented a real-time monocular 3D reconstruction system for mobile phone which used online incremental mesh generation for augmented reality application. For the 3D reconstruction process, they performed monocular depth estimation with a multi-view semi-global matching method followed by a depth refinement post-processing.…”