A YCSB Workload for Benchmarking Hotspot Object Behaviour in NoSQL Databases

Abstract

Many contemporary applications have to deal with unexpected spikes or unforeseen peaks in demand for specific data objectsso-called hotspot objects. For example in social networks, specific media items can go viral quickly and unexpectedly and therefore, properly provisioning for such behavior is not trivial. NoSQL databases are specifically designed for enhanced scalability, high availability, and elasticity to deal with increasing data volumes. Although existing performance benchmarking systems such as the Yahoo! Cloud Serving Benchmark (YCSB) provide support to test the performance properties of different databases under identical workloads, they lack support for testing how well these databases can cope with the abovementioned unexpected hotspot object behaviour. To address this shortcoming and fill the research gap, we present the design and implementation of a new YCSB workload that is rooted upon a formal characterization of hotspot-based spikes. The proposed workload implements the Pitman-Yor distribution and is configurable in a number of parameters such as spike probability and data locality. As such, it allows for more extensive experimental validation of database systems. Our functional validation illustrates how the workload can be used to effectively stress-test different types of databases and we present our comparative results of benchmarking two popular NoSQL databases that are Cassandra and MongoDB in terms of their response to spiked workloads.

Application-Based Benchmarking on Redis and MongoDB for Trip Planning using GTFS Data

“…Unfortunately, current database benchmarks like the Star Schema Benchmark (SSB) [22], TPC-DS Benchmark [23], and TPC-H [24] are inadequate for making these comparisons because the workloads depicted in these benchmarks do not accurately reflect how database queries are produced by user activities, such tools like Tableau [25] or Spotfire [20]. Some research benchmark database systems using interactive workload [26], [27], [28], [29], [30] while others use tools like Yahoo Cloud Service Benchmarking tool(YCSB) [31], [32], [33]and HammerDB [34]. In both approaches, a predefined static set of operations is to be performed as the workload; for example, a given workload can perform 1000 operations of 950 read and 50 updates.…”