OpenMP Parallelization and Optimization of Graph-Based Machine Learning Algorithms

Abstract

Abstract. We investigate the OpenMP parallelization and optimization of two novel data classification algorithms. The new algorithms are based on graph and PDE solution techniques and provide significant accuracy and performance advantages over traditional data classification algorithms in serial mode. The methods leverage the Nystrom extension to calculate eigenvalue/eigenvectors of the graph Laplacian and this is a self-contained module that can be used in conjunction with other graphLaplacian based methods such as spectral clustering. We use performance tools to collect the hotspots and memory access of the serial codes and use OpenMP as the parallelization language to parallelize the most timeconsuming parts. Where possible, we also use library routines. We then optimize the OpenMP implementations and detail the performance on traditional supercomputer nodes (in our case a Cray XC30), and predict behavior on emerging testbed systems based on Intel's Knights Corner and Landing processors. We show both performance improvement and strong scaling behavior. A large number of optimization techniques and analyses are necessary before the algorithm reaches almost ideal scaling.

Hyperspectral Image Classification Using Graph Clustering Methods

“…• These methods have been introduced in earlier conference papers [61,42] as serial implementations and as a parallel implementation [56] on a supercomputer, without online code. Here we develop new parallel versions for real time implementations on the IPOL server for both hyperspectral imagery and the non local means functional for segmentation of RGB images.…”

“…This motivated us to develop parallel implementations and optimizations of these two new algorithms [56]. In particular, for computations, we use an optimized implementation of the Nyström extension eigensolver on high performance computing systems.…”

Blind Hyperspectral Unmixing Based on Graph Total Variation Regularization

“…explored in our future work. For high performance computing applications, the Nyström loop can be optimized for specific architectures as in [57].…”

High Performance Computing and Real Time Software for High Dimensional Data Classification

“…Chapter 4 presents the work of one publication [107] which contains the full development of the hyperspectral image classification using two graph methods along with the parallelized codes and online demo. Chapter 5 discusses the publication [106] of detailed development of the parallelized algorithms using different high performance computing techniques.…”

“…Many successful machine learning methods have not been accelerated by high performance computing. This is a big opportunity and motivates us to develop parallel implementations and optimizations of the two classification algorithms [106] described in Chapter 3. We describe parallel implementations and optimizations of the new algorithms in Chapter 5.…”