Why and how to benchmark XML databases

Abstract

Benchmarks belong to the very standard repertory of tools deployed in database development. Assessing the capabilities of a system, analyzing actual and potential bottlenecks, and, naturally, comparing the pros and cons of different systems architectures have become indispensable tasks as databases management systems grow in complexity and capacity. In the course of the development of XML databases the need for a benchmark framework has become more and more evident: a great many different ways to store XML data have been suggested in the past, each with its genuine advantages, disadvantages and consequences that propagate through the layers of a complex database system and need to be carefully considered. The different storage schemes render the query characteristics of the data variably different. However, no conclusive methodology for assessing these differences is available to date.In this paper, we outline desiderata for a benchmark for XML databases drawing from our own experience of developing an XML repository, involvement in the definition of the standard query language, and experience with standard benchmarks for relational databases.

Pacd: A Bitmap-Based Framework for Processing XML Data

“…Additionally we implemented and evaluated our framework as a new XML-to-RDBMS mapping technique. Our experiment has demonstrated support for a wider range of XPath/XQuery (Schmidt et al 2001) queries, greater potential for optimisation through parallel processing of query components, reduced resource overheads, and reduced complexity of XML updates compared to alternatives in the literature.…”

“…We considered eight database functionalities (extracted from the XMark project (Schmidt et al 2001)), listed in Table2.…”

A Look Back on the XML Benchmark Project

“…In its current version, xmlgen requires less than 2 MB of mainmemory, and produces documents of sizes of 100 MB and 1 GB in 33.4 and 335.5 seconds, respectively (450MHz Pentium III). A more detailed description of the tool and downloads can be found on the project Web page [18].…”

XViz: A Tool for Visualizing XPath Expressions

“…We have experimented with XViz applied to three different workloads: the XMark benchmark [12], the XQuery Use Cases [6], and the XMach benchmark [4]. We describe here XMark only, which is shown in Fig.…”

“…The first occurs in XQueries 8,9,10,11,12,17 is connected by a dotted edge (i.e. ⊃) to the second one, which also occurs in XQuery 1.…”