Evaluation of SMP Shared Memory Machines for Use with In-Memory and OpenMP Big Data Applications

Abstract

Abstract-While distributed memory systems have shaped the field of distributed systems for decades, the demand for many-core shared memory resources is increasing. Symmetric Multiprocessor Systems (SMPs) have become increasingly important recently among a wide array of disciplines, ranging from Bioinformatics to astrophysics, and beyond. With the increase in big data computing, the size and scope of traditional commodity server systems is often outpaced. While some big data applications can be mapped to distributed memory systems found through many cluster and cloud technologies today, this effort represents a large barrier of entry that some projects cannot cross. Shared memory SMP systems look to effectively and efficiently fill this niche within distributed systems by providing high throughput and performance with minimized development effort, as the computing environment often represents what many researchers are already familiar with.In this paper we look at the use of two common shared memory systems, the ScaleMP vSMP virtualized SMP deployment at Indiana University, and the SGI UV architecture deployed at University of Arizona. While both systems are notably different in their design, their potential impact on computing are remarkably similar. As such, we look to compare each system first under a set of OpenMP threaded benchmarks via the SPEC group, and follow up with our experience using each machine for Trinity de-novo assembly. We find both SMP systems are well suited to support various big data applications, with the newer vSMP deployment often slightly faster, however certain caveats and performance considerations are necessary when considering such SMP systems.

Memory sharing for handling memory overload on physical machines in cloud data centers

“…ScaleMP vSMP aggregates multiple commodity x86-based PMs, such as a classic cluster, into a single virtualized system [35]. It runs a patched Linux operating system and uses InfiniBand-enabled network for interconnecting each PM [44]. TidalScale is another commercial software solution to aggregate computing resources of multiple PMs, which allows hot-swap individual PM for upgrade or repair while the mission-critical workload continues to run [4,31].…”