VFDS: Very fast database sampling system

Abstract

Abstract-In a wide range of application areas (e.g. data mining, approximate query evaluation, histogram construction), database sampling has proved to be a powerful technique. It is generally used when the computational cost of processing large amounts of information is extremely high, and a faster response with a lower level of accuracy for the results is preferred. Previous sampling techniques achieve this balance, however, an evaluation of the cost of the database sampling process should be considered. We argue that the performance of current relational database sampling techniques that maintain the data integrity of the sample database is low and a faster strategy needs to be devised. In this paper we propose a very fast sampling method that maintains the referential integrity of the sample database intact. The sampling method targets the production environment of a system under development, that generally consists of large amounts of data computationally costly to analyze. We evaluate our method in comparison with previous database sampling approaches and show that our method produces a sample database at least 300 times faster and with a maximum trade off of 0.5% in terms of sample size error.

VFDS

“…Based on JS, the authors of [6] propose the Linked Bernoulli Synopses (LBS) sampling method that aims to improve the query answering accuracy of the sample. We compare our proposed system against the two methods in [3].…”

“…In this paper, we propose to demonstrate VFDS [3], a novel automated system that generates a random sample database, given an original relational database and a sampling rate. Its objective is to generate fast the sample database, while preserving the referential integrity intact.…”

“…The system is a generic one and can be applied to any relational database, regardless of it containing a diamond pattern. VFDS performs between 300 and 1000 times faster than the compared techniques [3]. The high performance of the method makes VFDS a suitable candidate for real-time systems that require to quickly analyze a database.…”

“…The system proposed implemented the automated random sampling method VFDS [3]. Its objective is to maintain the referential integrity of the sampled data, while speeding up the sampling process.…”

“…Sampling tuples of t : The method randomly selects a percentage of the tuples of t from the original database. Experiments in [3] show that selecting the starting table with the most number of related tuples determines the lowest errors in terms of sample size and approximate query answering. [3] also presents the algorithm for selecting the starting table and for populating the sample database.…”