Self-Adapting Numerical Software and Automatic Tuning of Heuristics

Abstract

Self-Adapting Numerical Software (SANS) systems aim to bridge the knowledge gap that exists between the expertise of domain scientists, and the know-how that is needed to fulfill efficiently their computational demands. This know-how extends to algorith choice, computational grid utilization, and use of properly optimized kernels. A SANS system is a piece of meta software that mediates between the application program and the computational platform so that application scientists -with disparate levels of knowledge of algorithmic and programmatic complexities of the underlying numerical software -can easily realize numerical solvers and efficiently solve their problem.The main component of a SANS system is an Intelligent Agent that automates method selection based on data, algorithm and system attributes. The IA uses heuristics to make its decisions. In this paper we explain how the heuristics of the IA can be tuned over time by redundant testing and using the nature of many applications. IntroductionIn numerous technologically important areas, such as aerodynamics (vehicle design), electrodynamics (semiconductor device design), magnetohydrodynamics (fusion energy device design), and porous media (petroleum recovery), production runs on expensive, high-end systems last for hours or days, and a major portion of the execution time is usually spent inside of numerical routines, such as for the solution of large-scale nonlinear and linear systems that derive from discretized systems of nonlinear partial differential equations. These numerical parts of the code can contain a large number of tuning parameters, the choice of which greatly influences the efficiency of the total code, or can even make the difference between obtaining a solution and obtaining none.Such numerical concerns, however, are artifactual from the perspective of the scientific and engineering users, who are usually more concerned with modeling and discretization issues. The classic response to numerics was to encode the requisite mathematical, algorithmic and programming expertise into libraries that could be easily reused by a broad spectrum of domain scientists. However, in high-performance computing this solution is no longer sufficient. There is typically more than one algorithm for the stated purpose, and since several levels of algorithms are needed in a large-scale application; the different algorithms can have interlocking parameter settings, and the availability of parallel computing platforms influences algorithmic decisions. Since the difference in performance between an optimal choice of algorithm and hardware, and a less than optimal one, can span orders of magnitude, it is unfortunate that selecting the right solution strategy requires specialized knowledge of both numerical analysis and of computing platform characteristics. Our SANS system aims to assist the application user to navigate this maze of computational possibilities. *

Dense Linear Algebra over Word-Size Prime Fields

“…Here, a design using the finite field directly might improve the speed. -More generally, Self-adapting Software [Dongarra and Eijkhout 2003] would provide hybrid implementations with the best empirical thresholds. -The technique of wrapping BLAS becomes useless when finite fields are larger than the corresponding bound of feasibility (e.g.…”

A Standard and Software for Numerical Metadata

“…The first application is the development of the intelligent agent of a self-adapting numerical software system [Dongarra and Eijkhout 2003b;2003a]. Here, the exhaustive analyses of properties of a number of matrices are stored in a database for subsequent analysis, together with performance results, for instance, from solving linear systems with these matrices.…”

“…Such measurements have successfully been used in our SALSA system [Dongarra and Eijkhout 2003a;2003b]. For instance, a large difference between variability in rows and columns is a good indicator for asymmetric (left or right) scaling of the matrix.…”

Exploiting Capabilities of Many Core Platforms in Reservoir Simulation

“…We should remark that there have being previous works addressing this problem using a more sophisticate framework (Dongarra and Eijkhout, 2003;Mishev et al, 2008).…”