Rethinking large-scale Economic Modeling for Efficiency: Optimizations for GPU and Xeon Phi Clusters

Abstract

We propose a massively parallelized and optimized framework to solve high-dimensional dynamic stochastic economic models on modern GPU-and MIC-based clusters. First, we introduce a novel approach for adaptive sparse grid index compression alongside a surplus matrix reordering, which significantly reduces the global memory throughput of the compute kernels and maps randomly accessed data onto cache or fast shared memory. Second, we fully vectorize the compute kernels for AVX, AVX2 and AVX512 CPUs, respectively. Third, we develop a hybrid cluster oriented work-preempting scheduler based on TBB, which evenly distributes the time iteration workload onto available CPU cores and accelerators. Numerical experiments on Cray XC40 KNL "Grand Tave" and on Cray XC50 "Piz Daint" systems at the Swiss National Supercomputer Centre (CSCS) show that our framework scales nicely to at least 4,096 compute nodes, resulting in an overall speedup of more than four orders of magnitude compared to a single, optimized CPU thread. As an economic application, we compute global solutions to an annually calibrated stochastic public finance model with sixteen discrete, stochastic states with unprecedented performance.

High-Dimensional Dynamic Stochastic Model Representation

“…For the sake of brevity, we assume that the function value is zero on the domain's boundary. This is not a necessary condition, and it can be easily changed by augmenting the basis function (see, e.g., [47]). First, consider a one-dimensional domain, discretized with grid spacing h l = 2 −l .…”

“…When solving the system of equations at a given grid point, one needs to frequently interpolate from the function computed in the previous iteration step. These interpolation operations can account for up to 99% of the overall compute time for solving the system of equations [47]. These two impediments make it difficult to achieve an acceptable time-to-solution which can quickly reach the order of days on modern supercomputing facilities.…”

“…2 To deal with the ever-increasing complexity of state-of-the-art dynamic stochastic economic models, we propose in this work a generic, scalable, and flexible computational framework which can efficiently address the previously noted bottlenecks. Building on [47,6,5] and [12], we specifically contribute (i) an adaptive high-dimensional model representation scheme that is coupled with an adaptive sparse grid algorithm applicable for recursively formulated economic models that significantly reduces the number of grid points in the approximation and the time needed for each function evaluation, (ii) adaptivity criteria which can be used as an on-the-fly analysis tool elucidating the complexity of the model under consideration, (iii) a vectorized implementation for performant interpolation, and (iv) a hybrid parallelized time iteration solution framework fit for virtually any dynamic stochastic economic model. Finally (v), we deploy our solution framework at the Swiss National Supercomputing Center (CSCS) to solve highly nonlinear dynamic stochastic economic models of up to 300 dimensions globally.…”

Optimization of GPUs and Clusters for the Development of Large-scale Socio-economic Models on Supercomputers

“…пример [9]) для решения нелинейного набора уравнений, и при этом должны выполняться как можно быстрее, чтобы гарантировать быстрое принятия решения. Несмотря на наличие широко применяемого формата данных с плотной матрицей (см., например, [9]), для которого существуют высоко оптимизированные алгоритмы для выполнения задач, связанных с интерполяцией, необходимо внедрение нового, общего метода сжатия данных для АРС [20]. Данный метод позволит работать с 16 непрерывными измерениями, то есть с 59-мерными АРС одновременно.…”

Deep Structural Estimation: With an Application to Option Pricing

“…This methodology 1 In physics and engineering, Gaussian processes regression (see, e.g., Williams and Rasmussen, 2006;Tripathy, Bilionis, and Gonzalez, 2016;Bilionis and Zabaras, 2012a;Bilionis, Zabaras, Konomi, and Lin, 2013;Chen, Zabaras, and Bilionis, 2015), radial basis functions (Park and Sandberg, 1991), or relevance vector machines (Bilionis and Zabaras, 2012b) are often used to build surrogate models. More recently, following the rapid developments in the theory of stochastic optimization and artificial intelligence as well as the advances in computer hardware leading to the widespread availability of graphic processing units (GPUs; see, e.g., Scheidegger, Mikushin, Kubler, and Schenk (2018); Aldrich, Fernández-Villaverde, Gallant, and Rubio-Ramírez (2011), and references therein), researchers have turned their attention towards deep neural networks (see, e.g., Tripathy and Bilionis, 2018a;Liu, Borovykh, Grzelak, and Oosterlee, 2019a).…”