DD-αAMG on QPACE 3

Abstract

We describe our experience porting the Regensburg implementation of the DD-αAMG solver from QPACE 2 to QPACE 3. We first review how the code was ported from the first generation Intel Xeon Phi processor (Knights Corner) to its successor (Knights Landing). We then describe the modifications in the communication library necessitated by the switch from InfiniBand to Omni-Path. Finally, we present the performance of the code on a single processor as well as the scaling on many nodes, where in both cases the speedup factor is close to the theoretical expectations. IntroductionThe lattice QCD (LQCD) community has traditionally been an early adopter of new computing and network architectures. This typically requires major efforts porting simulation code or even communication libraries. The Regensburg lattice group (RQCD) has been involved in such efforts, as well as supercomputer development, for more than a decade. While the first computer in the QPACE series [1,2] was based on IBM's Cell processor and an FPGA-based custom interconnect, the subsequent machines are using Intel's Xeon Phi series with standard interconnects (see Sec. 2.1). To satisfy the increasing demands of the RQCD physics program we use a state-of-the-art method, DD-αAMG [3], to solve the discretized form of the Dirac equation. The high-performance implementation of this solver on QPACE 2 is described in [4][5][6][7]. The present contribution focuses on the software efforts we made to efficiently run this implementation on QPACE 3. This paper is structured as follows. In Sec. 2 we give an overview of QPACE 3 and highlight the differences to QPACE 2 in terms of processor and network. We discuss the network technology in some detail because it has changed rather drastically. In Sec. 3 we describe how our solver and our communication library were adapted to the new technologies. In Sec. 4 we present single-node and multi-node benchmarks of the solver on QPACE 3 and compare the results with numbers obtained on QPACE 2. In Sec. 5 we conclude and give an outlook on future work. QPACE 3 2.1 OverviewWhile QPACE 2 [8] is based on the Knights Corner (KNC) version of the Intel Xeon Phi processor series and an FDR InfiniBand network, its successor QPACE 3 utilizes the current Xeon Phi processor, Speaker,

Light-cone distribution amplitudes of pseudoscalar mesons from lattice QCD

“…We used a modified version of the Chroma [104] software package, along with the LibHadronAnalysis library and, depending on the target machine, either the multigrid DD-αAMG solver [105] implementation of refs. [106,107] or the domain decomposition solver of openQCD [55] (https://luscher.web.cern.ch/luscher/openQCD/). Most gauge ensembles have been generated by CLS (https://wiki-zeuthen.desy.de/CLS/) using openQCD.…”

Loss-Function Learning for Digital Tissue Deconvolution

“…Furthermore, we linked R with the Intel Math Kernel Library for threaded and vectorized matrix operations. We ran the algorithm on 25 nodes of our QPACE 3 machine Georg et al (2017) with 8 MPI tasks per node and 32 hardware threads per task, where each thread can use two AVX512 vector units. In 16 hours, 5086 iterations were finished, after which the loss (3) was stable to within 1%.…”

“…A more cost-and work-efficient alternative to single-cell assays is a combination of bulk tissue gene expression profiling with digital tissue deconvolution (DTD) (Lu et al, 2003;Abbas et al, 2009;Gong et al, 2011;Qiao et al, 2012;Altboum et al, 2014;Newman et al, 2015;Li et al, 2016). DTD addresses the following inverse problem: given the bulk gene expression profile y of a tissue, what is the cellular composition c of that tissue?…”

Lattice QCD on new chips: a community summary

“…An in-depth evaluation of the Intel Omni-Path network for LQCD applications and An implementation of the DD-α AMG multigrid solver on Intel Knights Landing: P. Georg and D. Richtmann presented the porting of their collaboration codebase onto the new QPACE3 machine, based on KNL nodes [24]. More specifically they presented their efforts in porting of the DD-αAMG solver.…”