Dataclay: A distributed data store for effective inter-player data sharing

Abstract

In the Big Data era, both the academic community and industry agree that a crucial point to obtain the maximum benefits from the explosive data growth is integrating information from different sources, and also combining methodologies to analyze and process it. For this reason, sharing data so that third parties can build new applications or services based on it is nowadays a trend. Although most data sharing initiatives are based on public data, the ability to reuse data generated by private companies is starting to gain importance as some of them (such as Google, Twitter, BBC or New York Times) are providing access to part of their data. However, current solutions for sharing data with third parties are not fully convenient to either or both data owners and data consumers. Therefore we present dataClay, a distributed data store designed to share data with external players in a secure and flexible way based on the concepts of identity and encapsulation. We also prove that dataClay is comparable in terms of performance with trendy NoSQL technologies while providing extra functionality, and resolves impedance mismatch issues based on the Object Oriented paradigm for data representation.This work has been supported by the Spanish Government (grant SEV2015-0493 of the Severo Ochoa Program), by the Spanish Ministry of Science and Innovation (contract TIN2015-65316) and by Generalitat de Catalunya (contract 2014-SGR-1051). Special thanks go to Dr. Oscar Romero (Universitat Politècnica de Catalunya) for providing helpful feedback on the paper.Peer ReviewedPostprint (published version

Predicting Access to Persistent Objects Through Static Code Analysis

“…Hibernate, DataNucleus, dataClay [11]), are storage systems that expose persistent data in the form of objects and relations between these objects. This structure is rich in semantics ideal for predicting access to persistent data [8] and has invited a significant amount of research due to the importance of these predictions in areas such as prefetching, cache replacement and dynamic data placement.…”

An Architecture for High Performance Computing and Data Systems Using Byte-Addressable Persistent Memory

“…We recognise the performance and functionality benefits that exploiting new storage technologies can bring to applications. We are therefore investigating the use of object stores, such as DAOS [21] and dataClay [22] and are porting them to the hardware architecture we are proposing, i.e. systems with distributed B-APM as the main storage hardware.…”

“…Another technology that is being widely investigated for improving performance and changing I/O functionality for applications is some form of object, key value, store [20] [21] [22]. These provide alternatives to file-based data storage, enabling data to be stored in similar formats or structures as those used in the application itself.…”

CAPre: Code-Analysis based Prefetching for Persistent Object Stores

“…Moreover, several POSs that support data distribution have been developed to accommodate the needs of parallel and distributed programming (e.g. Mneme [10], Nexus [11], Thor [12] and dataClay [13,14]).…”

“…We integrate CAPre into dataClay [14], a distributed POS, and run a series of experiments to measure the improvement in application performance that it can achieve. The experimental results indicate that using CAPre to prefetch objects from a POS can reduce execution times of applications, with the most significant gains observed in applications with complex data models and/or many collections of persistent objects.…”