NoSQL Database Performance Benchmarking - A Case Study

Abstract

This paper describes an experimental study regarding NoSQL database performance. Two NoSQL databases were considered (MongoDB and Cassandra), two different workloads (update heavy and mostly read) and several degrees of parallelism. The results refer to throughput performance metric. Statistical analysis performed is referring to throughput results. Conclusions confirm that MongoDB performs better than Cassandra in the context of a mostly read workload, while Cassandra outperformed MongoDB in the context of an update heavy workload where the number of operations was high.

Performance Benchmarking for NoSQL Database Management Systems

“…The case study presented in [1] analyzes the database performance metric called throughput, measured in number of operations/second. Other important and useful database performance metrics are latency (measured in number of microseconds/operation) and total runtime (the time necessary to run a certain number of database operations).…”

“…Experimental setting. I reproduced the experimental study presented in [1] using newer versions of database management systems and operating system without changing the hardware configuration. That experimental study involved three physical servers with the same hardware configuration.…”

“…YCSB version 0.12.0, MongoDB version 3.4.4 and Apache Cassandra 3.11.0 were each installed on its dedicated server. I followed the same benchmarking methodology and I performed all benchmarking tests in the same order and under the same conditions as those performed in the experimental study presented in [1]. The YCSB, MongoDB and Cassandra versions used in my experimental study were newer.…”

Using Latency Metrics in NoSQL Database Performance Benchmarking

“…The data set could fit within internal memory due to its size. The chosen workloads belong to the YCSB Core workloads set: Workload A (the ratio between read and write operations in the transaction set is 1 -50% updates, 50% reads), an update-heavy workload [20], and Workload B (the ratio between read and write operations in the transaction set is 1 19 -5% updates, 95% reads), a read-mostly workload [20]. An application that has a workload similar to Workload A could be a session store in which recent actions are recorded, while Workload B is similar to the workload of an application that involves photo tagging, as stated in [20].…”

“…This paper is the second describing a series of experimental studies, evaluating different performance indicators of two NoSQL database management systems: Cassandra and MongoDB. The current paper refers to two average latency measures, update latency and read latency, while the first one, [1], discussed the throughput averages. Latency indicators characterize the time needeed to perform a single operation/request (amount of time the request takes to complete), and throughput is related to the computational performance, measured in number of operations/requests performed per second.…”