Containers in HPC: a survey

Tesser, Rafael Keller; Borin, Edson

doi:10.1007/s11227-022-04848-y

Cited by 13 publications

(2 citation statements)

References 90 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, assessing emerging solutions that are extending containerization into specialized domains such as high-performance computing (HTC) environments [32,33] or addressing security challenges raised by containers [34] are also out of the scope of this work.…”

Section: Scope and Related Workmentioning

confidence: 99%

Toward Optimal Virtualization: An Updated Comparative Analysis of Docker and LXD Container Technologies

Silva,

Rafael,

Fonte

2024

Computers

View full text Add to dashboard Cite

Traditional hypervisor-assisted virtualization is a leading virtualization technology in data centers, providing cost savings (CapEx and OpEx), high availability, and disaster recovery. However, its inherent overhead may hinder performance and seems not scale or be flexible enough for certain applications, such as microservices, where deploying an application using a virtual machine is a longer and resource-intensive process. Container-based virtualization has received attention, especially with Docker, as an alternative, which also facilitates continuous integration/continuous deployment (CI/CD). Meanwhile, LXD has reactivated the interest in Linux LXC containers, which provides unique operations, including live migration and full OS emulation. A careful analysis of both options is crucial for organizations to decide which best suits their needs. This study revisits key concepts about containers, exposes the advantages and limitations of each container technology, and provides an up-to-date performance comparison between both types of containers (applicational vs. system). Using extensive benchmarks and well-known workload metrics such as CPU scores, disk speed, and network throughput, we assess their performance and quantify their virtualization overhead. Our results show a clear overall trend toward meritorious performance and the maturity of both technologies (Docker and LXD), with low overhead and scalable performance. Notably, LXD shows greater stability with consistent performance variability.

show abstract

Section: Scope and Related Workmentioning

confidence: 99%

Toward Optimal Virtualization: An Updated Comparative Analysis of Docker and LXD Container Technologies

Silva,

Rafael,

Fonte

2024

Computers

View full text Add to dashboard Cite

show abstract

“…Furthermore, this implementation decouples TSP solutions from the execution of metaheuristics. Therefore, different distributed metaheuristics can be dynamically applied or executed asynchronously in the framework using continuous integration tools, such as Jenkins [49].…”

Section: Distributed Ctsp Solving Frameworkmentioning

confidence: 99%

A Cloud Container-Based Distributed Solving Approach to Superscale Colored Traveling Salesman Problems

Lin,

Li,

2023

Preprint

View full text Add to dashboard Cite

The colored traveling salesman problem (CTSP) generalizes the well-known multiple traveling salesman problem (MTSP) by utilizing colors to describe the accessibility of cities to individual salesmen. Many serial algorithms have been developed to solve CTSP instances. This work presents a distributed solving method for CTSP for the first time. First, a cloud container-based distributed solving framework for CTSP is designed and implemented in a pipeline style. A superscale CTSP can be decomposed into many TSPs to solve in parallel to reduce computational complexity dramatically. Second, following the framework, we develop a distributed Delaunay-triangulation-based variable neighborhood search (DDVNS) algorithm. In DDVNS, a two-stage initialization is exploited to generate each TSP solution. Then, Delaunay-triangulation-based variable neighborhood search (DVNS) is applied to search for the local TSP optima within a neighborhood delicately. In addition, the solutions are improved further by reallocating the multicolor cities and repeating the search progress. Finally, the extensive experiments show that DDVNS outperforms the state-of-the-art serial algorithms regarding search efficiency and solution quality. Notably, We can achieve a satisfactory solution in a superscale case of 16 salesmen and 160000 cities within 15 minutes, which breaks the existing solving record of CTSPs.

show abstract

Parsing Millions of DNS Records Per Second

Koekkoek,

Lemire

2024

Softw Pract Exp

View full text Add to dashboard Cite

ObjectivesTo enhance the throughput of DNS parsing by addressing the computational expense of processing large plain text DNS zone files.To specifically increase the speed of parsing DNS zone files compared to existing state‐of‐the‐art parsers.MethodDevelopment of a new approach named simdzone for DNS parsing.Utilization of data parallelism through Single Instruction Multiple Data (SIMD) instructions available on modern processors to accelerate parsing operations.ResultThe simdzone approach significantly increased parsing speeds, being several times faster than the parsers in Knot DNS and NLnet Labs' Name Server Daemon (NSD).The software library developed from this approach was integrated into NSD, replacing its previous parser.ConclusionThe implementation of SIMD‐based data parallelism in DNS parsing provides a substantial performance improvement, making it a viable solution for handling large DNS zone files more efficiently. This not only reduces processing time but also enhances the overall functionality of DNS services.

show abstract

Containers in HPC: a survey

Cited by 13 publications

References 90 publications

Toward Optimal Virtualization: An Updated Comparative Analysis of Docker and LXD Container Technologies

Toward Optimal Virtualization: An Updated Comparative Analysis of Docker and LXD Container Technologies

A Cloud Container-Based Distributed Solving Approach to Superscale Colored Traveling Salesman Problems

Parsing Millions of DNS Records Per Second

Contact Info

Product

Resources

About