Complex inference from simulation ensembles used in uncertainty quantification leads to twin computational challenges of managing large amount of data and performing CPU-intensive computing. While algorithmic innovations using surrogates, localization and parallelization can make the problem feasible, one still has very large data and compute tasks. The problem of dealing with large data gets compounded when data warehousing and data mining are intertwined with computationally expensive tasks. We present here an approach to solving this problem by using a mix of hardware suitable for each task in a carefully orchestrated workflow. The computing environment is essentially an integration of Netezza database and high-performance cluster. It is based on the simple idea of segregating the data-intensive and computeintensive tasks and assigning the right architecture for them. We present here the layout of the computing model and the new computational scheme adopted to generate probabilistic hazard maps.
Uncertainty Quantification(UQ) using simulation ensembles leads to twin challenges of managing large amount of data and performing cpu intensive computing. While algorithmic innovations using surrogates, localization and parallelization can make the problem feasible one still has very large data and compute tasks. Such integration of large data analytics and computationally expensive tasks is increasingly common. We present here an approach to solving this problem by using a mix of hardware and a workflow that maps tasks to appropriate hardware. We experiment with two computing environments -the first is an integration of a Netezza data warehouse appliance and a high performance cluster and the second a hadoop based environment. Our approach is based on segregating the data intensive and compute intensive tasks and assigning the right architecture to each. We present here the computing models and the new schemes in the context of generating probabilistic hazard maps using ensemble runs of the volcanic debris avalanche simulator TITAN2D and UQ methodology.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.