Using evolving storage structures for data storage

Abstract

Different data storage structures suit different data management scenarios, therefore, a universal data storage structure is not possible. Furthermore, each data storage structure has different resource consumption because of their unique execution complexity. Self-tuning data management systems need a mechanism to select appropriate data storage structure and to adapt it with changing data management requirements. We propose Evolving Storage Structures for self-tuning data management system that supports customization and adaptation with changing data management needs.

ECOS: Evolutionary Column-Oriented Storage

“…ECOS enforces that each storage structure should be atomic and should be directly accessible using an access API. The reason for this approach is that small storage structures consume less memory and generate reduced binary size for small data management [28]. If we can use them directly, than there is no reason to use them as part of complex storage structures (we use storage structure as a common term for both data storage structure and index storage structure), such as B+-Tree or T-Tree; avoiding the overheads of complexity associated with these storage structures.…”

“…The fine-grained customization is supported at the table-level and column-level according to the recommendations/results from [2,12,17]. We also identified the need to autonomically change the existing data and index storage structure to more appropriate ones with the changing data management needs based on our previously published results in [28]. We named our solution Evolutionary Column-Oriented Storage (ECOS), which is based on the existing Decomposed Storage Model (DSM) [12] with the novel capability of automatically evolving the internal data and index storage structures for each column with the growth of data.…”