MemepiC: Towards a Unified In-Memory Big Data Management System

Cai, Qingchao; Zhang, Hao; Guo, Wentian; Chen, Gang; Ooi, Beng Chin; Tan, Kian-Lee; Wong, Weng-Fai

doi:10.1109/tbdata.2017.2789286

Cited by 17 publications

(14 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The performance results are consistent with the published results[15] with a larger cluster of higher-end servers 6. This is the extra bonus to FaRM, as we do not optimize for GAM.…”

supporting

confidence: 90%

“…After receiving both the reply from the owner node (4.1) and the acknowledgement from the home node (6), the request node nr can now proceed by logging the new data, updating the cache line and setting its state as "Dirty" (7). It should be noted the operations of (7) can be performed only after both messages, i.e., (4.1) and (6), have been received. Otherwise, if the request node nr were to perform (7) immediately after receiving (4.1), it may give up the ownership (e.g., due to an eviction) before the home node n h receives the ownership transfer message from the owner node no (4.2), in which case the home node will believe the request node gets the ownership when in fact it no longer has the ownership.…”

Section: Remote Writementioning

confidence: 99%

“…FaRM does not perform well when distribution ratio is low, which is mainly due to the fact that there is still some data (e.g., ITEM table) that cannot be co-partitioned with other tables (e.g., STOCK table) , even though we have already manually co-partitioned the index for FaRM 6 . L-Store performs well when there are few distributed transactions, but its performance drops significantly even when there are only a small amount of distributed transactions, which is mainly because of the network overhead in L-Store.…”

Section: Distributed Transaction Processingmentioning

confidence: 99%

“…Shared-nothing programming model has been widely used in distributed computing for its scalability. One popular example is distributed key-value store [6,21,33,36,44], which uses key-value APIs (e.g., Put and Get) to access remote data. In comparison, the shared-memory model that is able to accss remote data via memory semantics renders a unified global memory abstraction very attractive for distributed computing, since it not only unifies global data access, but also, more importantly, enables users to view distributed computing nodes as a powerful server with a single unified memory space and hence develop distributed applications in the same way as they do multi-threaded programming.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Efficient distributed memory management with RDMA and caching

et al. 2018

Self Cite

View full text Add to dashboard Cite

Recent advancements in high-performance networking interconnect significantly narrow the performance gap between intra-node and inter-node communications, and open up opportunities for distributed memory platforms to enforce cache coherency among distributed nodes. To this end, we propose GAM, an efficient distributed in-memory platform that provides a directory-based cache coherence protocol over remote direct memory access (RDMA). GAM manages the free memory distributed among multiple nodes to provide a unified memory model, and supports a set of userfriendly APIs for memory operations. To remove writes from critical execution paths, GAM allows a write to be reordered with the following reads and writes, and hence enforces partial store order (PSO) memory consistency. A lightweight logging scheme is designed to provide fault tolerance in GAM. We further build a transaction engine and a distributed hash table (DHT) atop GAM to show the ease-of-use and applicability of the provided APIs. Finally, we conduct an extensive micro benchmark to evaluate the read/write/lock performance of GAM under various workloads, and a macro benchmark against the transaction engine and DHT. The results show the superior performance of GAM over existing distributed memory platforms.

show abstract

“…The performance results are consistent with the published results[15] with a larger cluster of higher-end servers 6. This is the extra bonus to FaRM, as we do not optimize for GAM.…”

supporting

confidence: 90%

Section: Remote Writementioning

confidence: 99%

Section: Distributed Transaction Processingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient distributed memory management with RDMA and caching

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [17], the author tells us that because of less expensive and faster processing in DRAM in-memory data management systems have gained a lot of attention. The author proposed an innovative in-memory data management system named as MemepiC.…”

Section: A Research Paper Summariesmentioning

confidence: 99%

Fast and Efficient In-Memory Big Data Processing

Malik¹,

Maryam²,

Khalid³

et al. 2019

IJACSA

View full text Add to dashboard Cite

With the passage of time, the data is growing exponentially and the mostly endured areas are social media networks, media hosting applications, and servers. They have thousands of Tera-bytes of data and the efficient systems, however, they are as yet confronting issue to oversee such volume of information and its size is growing each day. Data systems retrieve information with less time of In-memory. Instead of each factor data systems are required to define good usage of cache and fast memory access with help of optimization. The proposed technique to solve this problem can be the optimal indexing technique with better and efficient utilization of Cache and having less overhead of DRAM with the goal that energy can also be saved for the high-end servers.

show abstract