Evaluating the performance of object-oriented database systems by means of a concrete application

Abstract

Because of the increasing use of object-oriented database systems, special benchmarks have been developed to analyze performance. The benchmarks show unanimously that object-oriented systems are wellsuited for applications that manage complex structures. Particularly, object-oriented database systems provide higher performance than relational ones. This is proven by synthetic schemas and elementary operations such as inserts and traversals, which are supposed to be characteristic for a lot of objectoriented applications. This experience report presents the results of a real application-specific investigation, comparing three commercial object-oriented database systems. The tests we performed took the elapsed times of an application "as a whole." In spite of having characteristics similar to standard benchmarks, we obtained huge performance differences. To put it in a nutshell, standard benchmarks are not always meaningful and should not cause one to jump to conclusions.

Comparing the Performance of Object and Object Relational Database Systems on Objects of Varying Complexity

“…The complex search is also very fast in ODBMSs. The study by Hohenstein et al [9] is similar to our work in that it evaluates the performance of DBMSs by means of a concrete object-oriented application. However they use a real dataset for their experiments while we use an artificial dataset.…”

“…For example, [9] compared the performance of various commercial ODBMSs. More recently, [11] compared the performance of ODBMSs and Object Relational Mapping (ORM) tools.…”

“…Hohenstein et al [9] performed an application-specific comparison of the three best known commercial ODBMSs. The goal of their evaluation was to create a realistic test for ODBMSs, allowing for a fair and precise comparison of performance.…”

“…The researchers also compared the ODBMSs with the original, real-world relational system; however they believed that this comparison is vague because the times for the RDBMS were measured while concurrent processes may influence locking and elapsed times. In the study by Hohenstein et al [9] the ODBMSs remain anonymous and they are introduced as ODBMS1, 2 and 3. Each ODBMS has been tuned heavily according to its specific architecture.…”

“…Hohenstein et al [9] concluded that traversals of relationships are much faster in the page server ODBMSs than in related SQL queries. Since ODBMSs do update operations in the primary memory and update in the server and disk is postponed to the commit, this results in slower update operations by ODBMSs compared to RDBMS.…”