Locabus: A Kernel to Kernel Communication Channel for Cluster Computing

Werstein, P.; Pethick, M.; Huang, Zhiyi

doi:10.1007/978-3-540-30501-9_98

Cited by 5 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LocaSwap [7] provides remote memory based on Ethernet network in kernel space called LocaBus [8]. LocaSwap is the virtual RamDisk implemented as a normal block disk driver.…”

Section: Related Workmentioning

confidence: 99%

A Remote Memory System for High Performance Data Processing

Choi¹,

Kim²,

Bae³

2015

IJFCC

View full text Add to dashboard Cite

Abstract-Application's memory footprints are growing exponentially due to an increase in their data set and additional software layer. Modern RDMA capable networks such as InfiniBand and Myrinet with low latency and high bandwidth provide us a new vision to utilize remote memory. Remote idle memory can be exploited to improvement performance of memory intensive applications on individual nodes. Network swapping will be faster than traditional swapping to local disk. In this paper, we design a remote memory system for remote memory utilization in InfiniBand clusters. We present the architecture, communication method and algorithm of InfiniBand Block Device (IBD), which is implemented as loadable kernel module for version 3.5.0-45 of the Linux kernel. Especially, we discuss design issues transfer pages to remote memory. Our experiments show that IBD can bring more performance gain for applications whose working sets are larger than the local memory on a node but smaller than idle memory available on the cluster.Index Terms-Remote memory, distributed memory, swapping, cluster system, InfiniBand. I. INTRODUCTIONApplication's memory footprints are growing exponentially due to an increase in their data set and additional software layer. This memory requirement outpaces the growth in the capacity of current memory modules. Traditionally magnetic disks are used as the backing store for virtual memory. However the overall performance of applications is degraded due to the low speed of disks when applications need more memory than is physically available. In addition, since processor performance improves at a higher rate than disk seek latency, the cost of a disk access continues to increase with time [1].Modern networking technologies such as Infiniband and Myrinet with low latency of a few microseconds and high bandwidth of up to 10 Gbps provide us a new vision to utilize remote memory for local system performance improvement. It is clear that paging to idle memory is faster than paging to disk because network ram eliminates the physical seek time and the bandwidth of network connections is increasing faster than the bandwidth to disk .Several Remote memory mapping techniques deal with the remote memory as an extension to the local memory space. These techniques require inflexible malloc-like APIs and recompilation of the existing applications. Remote memory swapping techniques deal with the remote memory as a swap device. The aim of these techniques is partially to fill the performance gap between local memory and hard disk without modifying the OS or the running applications.Especially cluster systems, that encapsulate hundreds or even thousands of independent computing nodes within a single platform, have an imbalance of memory usage across different computing nodes. Therefore large amounts of idle cluster memory are almost always available for remote paging [4], [5]. Remote paging adds remote memory between main memory and disk in the local memory hierarchy. This caching technology provides an efficient way to boost...

show abstract

“…LocaSwap [7] provides remote memory based on Ethernet network in kernel space called LocaBus [8]. LocaSwap is the virtual RamDisk implemented as a normal block disk driver.…”

Section: Related Workmentioning

confidence: 99%

A Remote Memory System for High Performance Data Processing

Choi¹,

Kim²,

Bae³

2015

IJFCC

View full text Add to dashboard Cite

show abstract

“…Many protocols have been proposed for wide area networks. Some implementations over Ethernet-connected networks can be found in [13], [14]. However, to the best of our knowledge, not many authors integrated reliable multicast protocols into multi-agent based simulation, which is our main contribution in this paper.…”

Section: Introductionmentioning

confidence: 99%

Reliable Multicast Protocol in Distributed Simulation for Multi-agent Systems

Hai

2009

2009 IEEE-RIVF International Conference on Computing and Communication Technologies

View full text Add to dashboard Cite

Using computational clusters to simulate multi-agent systems has attracted a considerable amount of interest in recent years because of its ability to perform investigations about large complex systems. Many factors such as the load balancing among the nodes, the communication latencies or the synchronizing the logical processes, however, can impact on its performance. This paper focuses on the minimization of communication costs by proposing using reliable multicast mechanism for reducing communication latency problem. This promises to give an improvement in its performance.

show abstract