The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

No abstract

We develop speed and efficiency improvements to a three-dimensional (3D) digital volume correlation (DVC) algorithm, which measures displacement and strain fields throughout the interior of a material. Our goal is to perform DVC with resolution comparable to that achieved in 2D digital image correlation, in time that is commensurate with the image acquisition time. This would represent a significant improvement over the current state-of-the-art available in the literature. Using an X-ray micro-CT scanner, we can resolve features at the 5 micron scale, generating 3D images with up to 36 billion voxels. We compute twelve degrees-of-freedom at each correlation point and utilize tricubic spline interpolation to achieve high accuracy. We improve the algorithm's speed and robustness through an improved coarse search, efficient implementation of spline interpolation, and using smoothing splines to address noisy image data. For DVC, the volume of data, number of correlation points, and work to solve each correlation point grow cubically. We therefore employ parallel computing to handle this tremendous increase in computational and memory requirements. We demonstrate the application M. Gates (B, SEM member) · M.T. Heath

The computation of the singular value decomposition, or SVD, has a long history with many improvements over the years, both in its implementations and algorithmically. Here, we survey the evolution of SVD algorithms for dense matrices, discussing the motivation and performance impacts of changes. There are two main branches of dense SVD methods: bidiagonalization and Jacobi. Bidiagonalization methods started with the implementation by Golub and Reinsch in Algol60, which was subsequently ported to Fortran in the EIS-PACK library, and was later more efficiently implemented in the LINPACK library, targeting contemporary vector machines. To address cache-based memory hierarchies, the SVD algorithm was reformulated to use Level 3 BLAS in the LAPACK library. To address new architectures, ScaLAPACK was introduced to take advantage of distributed computing, and MAGMA was developed for accelerators such as GPUs. Algorithmically, the divide and conquer and MRRR algorithms were developed to reduce the number of operations. Still, these methods remained memory bound, so two-stage algorithms were developed to reduce memory operations and increase the computational intensity, with efficient implementations in PLASMA, DPLASMA, and MAGMA. Jacobi methods started with the two-sided method of Kogbetliantz and the one-sided method of Hestenes. They have likewise had many developments, including parallel and block versions and preconditioning to improve convergence. In this paper, we investigate the impact of these changes by testing various historical and current implementations on a common, modern multicore machine and a distributed computing platform. We show that algorithmic and implementation improvements have increased the speed of the SVD by several orders of magnitude, while using up to 40 times less energy.