Molecular Simulation Workflows as Parallel Algorithms: The Execution Engine of Copernicus, a Distributed High-Performance Computing Platform

Abstract

Computational chemistry and other simulation fields are critically dependent on computing resources, but few problems scale efficiently to the hundreds of thousands of processors available in current supercomputersparticularly for molecular dynamics. This has turned into a bottleneck as new hardware generations primarily provide more processing units rather than making individual units much faster, which simulation applications are addressing by increasingly focusing on sampling with algorithms such as free-energy perturbation, Markov state modeling, metadynamics, or milestoning. All these rely on combining results from multiple simulations into a single observation. They are potentially powerful approaches that aim to predict experimental observables directly, but this comes at the expense of added complexity in selecting sampling strategies and keeping track of dozens to thousands of simulations and their dependencies. Here, we describe how the distributed execution framework Copernicus allows the expression of such algorithms in generic workflows: dataflow programs. Because dataflow algorithms explicitly state dependencies of each constituent part, algorithms only need to be described on conceptual level, after which the execution is maximally parallel. The fully automated execution facilitates the optimization of these algorithms with adaptive sampling, where undersampled regions are automatically detected and targeted without user intervention. We show how several such algorithms can be formulated for computational chemistry problems, and how they are executed efficiently with many loosely coupled simulations using either distributed or parallel resources with Copernicus.

GROMACS: High performance molecular simulations through multi-level parallelism from laptops to supercomputers

“…A typical 150,000-atom system has about thirty million particle-particle interactions per MD step, which will not scale to a million-core system because communication and book-keeping costs will dominate. The Copernicus ensemble framework has been developed alongside GROMACS 5, to serve this need and scale to tens of thousands of simulations [29]. It currently supports free-energy calculations, Markov state modeling, and the string method using swarms [30] (http://copernicus.gromacs.org).…”

Molecular dynamics simulations of membrane proteins and their interactions: from nanoscale to mesoscale

“…Ongoing advances in for example the development of simulation codes to efficiently exploit very large scale computing resources, including CPU/GPU combination [60], and in methods for setup and analysis of complex simulation systems [61] enable molecular simulations of membranes to achieve length scales of several hundred nanometers, thus permitting direct comparison with cell membrane imaging by cryo-electron tomography and by superresolution optical microscopies. Using these approaches, simulations of for example whole virus particles and subcellular organelles become feasible.…”

Continuous Evaluation of Ligand Protein Predictions: A Weekly Community Challenge for Drug Docking

“…52 To address these issues, the community has seen a dramatic uptick in the use and availability of automated workflows that clearly memorialize a particular experiment and provide different approaches for their execution and deployment. [53][54][55][56][57] Further, although many performance comparisons have been published, the results can be difficult to interpret. 47,[58][59][60][61][62][63][64][65][66][67][68] For example, new docking algorithms are frequently published along with a comparison against existing methods, but this comparison is often secondary to the description of the new algorithm, and hence not fully developed.…”