Supercomputing centers are seeing increasing demand for userdefined software stacks (UDSS), instead of or in addition to the stack provided by the center. These UDSS support user needs such as complex dependencies or build requirements, externally required configurations, portability, and consistency. The challenge for centers is to provide these services in a usable manner while minimizing the risks: security, support burden, missing functionality, and performance. We present Charliecloud, which uses the Linux user and mount namespaces to run industry-standard Docker containers with no privileged operations or daemons on center resources. Our simple approach avoids most security risks while maintaining access to the performance and functionality already on offer, doing so in just 800 lines of code. Charliecloud promises to bring an industry-standard UDSS user workflow to existing, minimally altered HPC resources.

HPC centers are facing increasing demand for greater software flexibility to support faster and more diverse innovation in computational scientific work. Containers, which use Linux kernel features to allow a user to substitute their own software stack for that installed on the host, are an increasingly popular method to provide this flexibility. Because standard container technologies such as Docker are unsuitable for HPC, three HPC-specific technologies have emerged: Charliecloud, Shifter, and Singularity. A common concern is that containers may introduce performance overhead. To our knowledge, no comprehensive, rigorous, HPC-focused assessment of container performance has previously been performed. Our present experiment compares the performance of all three HPC container implementations and bare metal on multiple dimensions using industry-standard benchmarks (SysBench, STREAM, and HPCG). We found no meaningful performance differences between the four environments, with the possible exception of modest variation in memory usage. These results suggest that HPC users should feel free to containerize their applications without concern about performance degradation, regardless of the container technology used. It is an encouraging development towards greater adoption of userdefined software stacks to increase the flexibility of HPC systems.

Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Container technologies like Docker bring great benefits to the application development, build and deployment processes. While much cloud computing infrastructure is already designed to support Docker, little work has been done to support production Docker deployment on HPC systems. In this work, we propose a Docker-enabled Build and Execution Environment (BEE) for HPC systems and detail a standard backend for BEE using virtual machines, the BEE-VM. This brings many of the benefits of Docker to existing HPC machines in user-space without the requirement of specialized pre-installed software and with no system administrator configuration. We show that current HPC application can be easily configured to run within BEE, eliminating the need to reconfigure and rebuild applications for different systems while preserving comparable performance.

No abstract

No abstract