On Using Linux Kernel Huge Pages with FLASH, an Astrophysical Simulation Code

Calder, A. C.; Feldman, Catherine; Siegmann, Eva; Dey, John; Curtis, Anthony; Chheda, Smeet; Harrison, Robert J.

doi:10.1109/cluster51413.2022.00070

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, these counters represent the behavior in that specific module, rather than the software as a whole, while the timers show the full runtime. As expected and seen in our last study [6], in both cases the hardware cycles, MMB, and overall runtime are about the same when using hp, thp, or no hp. However, using hp drastically decreases the DTLB miss rate, while using thp does not have as much of an effect.…”

Section: Resultssupporting

confidence: 88%

“…This work extends our initial study of using hugepages with just the Fujitsu compiler, which demonstrated that hugepages did not provide a significant speedup [6]. Our speculation was that TLB misses might not make much of a difference because the A64FX has hardware to ameliorate the cost of TLB misses by avoiding OS calls, or because the FLASH data access patterns do not trigger a performance penalty.…”

Section: Previous Work With Hugepagesmentioning

confidence: 56%

See 1 more Smart Citation

A Further Study of Linux Kernel Hugepages on A64FX with FLASH, an Astrophysical Simulation Code

Feldman,

Chheda,

Calder

et al. 2023

Practice and Experience in Advanced Research Computing

Self Cite

View full text Add to dashboard Cite

We present an expanded study of the performance of FLASH when using Linux Kernel Hugepages on Ookami, an HPE Apollo 80 A64FX platform. FLASH is a multi-scale, multi-physics simulation code written principally in modern Fortran and makes use of the PARAMESH library to manage a block-structured adaptive mesh. Our initial study used only the Fujitsu compiler to utilize standard hugepages (hp), but further investigation allowed us to utilize hp for multiple compilers by linking to the Fujitsu library libmpg and transparent hugepages (thp) by enabling it at the node level. By comparing the results of hardware counters and in-code timers, we found that hp and thp do not significantly impact the runtime performance of FLASH. Interestingly, there is a significant reduction in the TLB misses, differences in cache and memory access counters, and strange behavior is observed when using thp. CCS CONCEPTS• Computer systems organization → Parallel architectures; • Computing methodologies → Modeling and simulation; Massively parallel and high-performance simulations; • Applied computing → Physics; Astronomy.

show abstract

Section: Resultssupporting

confidence: 88%

Section: Previous Work With Hugepagesmentioning

confidence: 56%

A Further Study of Linux Kernel Hugepages on A64FX with FLASH, an Astrophysical Simulation Code

Feldman,

Chheda,

Calder

et al. 2023

Practice and Experience in Advanced Research Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…But because we also found that memory access is only 20-40% of the runtime (not illustrated in the plots), we conclude that the increased bandwidth can't completely account for the speedup. Complete results of our exploration of hp may be found in [27,28].…”

Section: Huge Pages and Compiler Comparisonmentioning

confidence: 99%

Ookami: An A64FX Computing Resource

Calder,

Siegmann,

Feldman

et al. 2024

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

We present a look at Ookami, a project providing community access to a testbed supercomputer with the ARM-based A64FX processors developed by a collaboration between RIKEN and Fujitsu and deployed in the Japanese supercomputer Fugaku. We provide an overview of the project and details of the hardware, and describe the user base and education/training program. We present highlights from previous performance studies of two astrophysical simulation codes and present a strong scaling study of a full 3D supernova simulation as an example of the the machine’s capability.

show abstract

Benchmarking with Supernovae: A Performance Study of the FLASH Code

Martin,

Feldman,

Calder

et al. 2024

Practice and Experience in Advanced Research Computing 2024: Human Powered Computing

View full text Add to dashboard Cite

Astrophysical simulations are computation, memory, and thus energy intensive, thereby requiring new hardware advances for progress. Stony Brook University recently expanded its computing cluster "SeaWulf" with an addition of 94 new nodes featuring Intel Sapphire Rapids Xeon Max series CPUs. We present a performance and power efficiency study of this hardware performed with FLASH: a multi-scale, multi-physics, adaptive mesh-based software instrument. We extend this study to compare performance to that of Stony Brook's Ookami testbed which features ARM-based A64FX-700 processors, and SeaWulf's AMD EPYC Milan and Intel Skylake nodes. Our application is a stellar explosion known as a thermonuclear (Type Ia) supernova and for this 3D problem, FLASH includes operators for hydrodynamics, gravity, and nuclear burning, in addition to routines for the material equation of state. We perform a strong-scaling study with a 220 GB problem size to explore both single-and multi-node performance. Our study explores * Both authors contributed equally to this research.

show abstract

On Using Linux Kernel Huge Pages with FLASH, an Astrophysical Simulation Code

Cited by 3 publications

References 0 publications

A Further Study of Linux Kernel Hugepages on A64FX with FLASH, an Astrophysical Simulation Code

A Further Study of Linux Kernel Hugepages on A64FX with FLASH, an Astrophysical Simulation Code

Ookami: An A64FX Computing Resource

Benchmarking with Supernovae: A Performance Study of the FLASH Code

Contact Info

Product

Resources

About