Enabling and scaling biomolecular simulations of 100 million atoms on petascale machines with a multicore-optimized message-driven runtime

Abstract

A 100-million-atom biomolecular simulation with NAMD is one of the three benchmarks for the NSF-funded sustainable petascale machine. Simulating this large molecular system on a petascale machine presents great challenges, including handling I/O, large memory footprint and getting good strong-scaling results. In this paper, we present parallel I/O techniques to enable the simulation. A new SMP model is designed to efficiently utilize ubiquitous wide multicore clusters by extending the CHARM++ asynchronous message-driven runtime. We exploit node-aware techniques to optimize both the application and the underlying SMP runtime. Hierarchical load balancing is further exploited to scale NAMD to the full Jaguar PF Cray XT5 (224,076 cores) at Oak Ridge National Laboratory, both with and without PME full electrostatics, achieving 93% parallel efficiency (vs 6720 cores) at 9 ms per step for a simple cutoff calculation. Excellent scaling is also obtained on 65,536 cores of the Intrepid Blue Gene/P at Argonne National Laboratory.

Automated Load Balancing Invocation Based on Application Characteristics

“…It simulates the behavior of atoms based on the Lennard-Jones potential, which is an effective potential that describes the interaction between two uncharged molecules or atoms. The computation performed in this code mimics the short-range non-bonded force calculation in NAMD [7], an application widely used by biophysicists, that won the Gordon Bell award.…”

“…Therefore, load balancer can use the instrumented load information to make load balancing decisions. The key advantage of this approach is that it is application independent, and it has been shown to be effective for a large class of applications, such as NAMD [7], ChaNGa [8] and Fractography3D [9].…”

Overcoming the Scalability Challenges of Epidemic Simulations on Blue Waters

“…We leverage CHARM++'s SMP machine layer [9], which instead of creating one OS process per core of an n core node, the runtime creates k OS processes per node, such that k<n and n k is an integer. This allows chares within a process to leverage more efficient intra-node communication via shared memory for the following benefits: (i) inter-thread communication can be implemented with direct memory copy, (ii) the communication thread minimizes the interference between application compute functions and communication, (iii) sharing of read-only data across all threads reduces memory consumption.…”

Optimizing fine-grained communication in a biomolecular simulation application on Cray XK6

“…In this paper, we extended the work to support the SMP (multi-threaded) execution mode [13], [12], which is tuned for running on multicore-based parallel machines.…”

“…In our earlier work, we have shown how NAMD scales to 64K cores of BG/P, and 224k cores on Jaguar XT5. Table I presents the performance of NAMD running a 100-million-atom system on Jaguar XT5 in 2011 [12]. It can be seen that the speedup starts to falter beyond 64K cores.…”