On RDBMS-Integrated Disk-Based Architecture for Managing Massive Dormant Data in a Compressed Format

Abstract

Managing dormant data in very large databases (VLDB) is about striking a balance between cost and performance. While magnetic tapes have long been a standard answer, the downsides limit the applicability for all VLDB. This paper describes a class of engineering and scientific data management applications that need a cost-effective solution with better support for updating and lower granularity of data access. To meet the needs of such applications, it then introduces an architectural extension to RDBMS. In the architecture, data is compressed using custom compression techniques, stored outside an RDBMS, and made available through views using SQL Table functions. This architecture is applicable to many industrial RDBMS, and it offers both linear speed-up and scale-out. The paper then discusses preliminary findings from a prototype implementation on Oracle RDBMS.