Automated construction of JavaScript benchmarks

Abstract

JavaScript is a highly dynamic language for web-based applications. Innovative implementation techniques for improving its speed and responsiveness have been developed in recent years. Industry benchmarks such as WebKit SunSpider are often cited as a measure of the efficacy of these techniques. However, recent studies have shown that these benchmarks fail to accurately represent the dynamic nature of modern JavaScript applications, and so may be poor predictors of real-world performance. Worse, they may guide the development of optimizations which are unhelpful for real applications. Our goal is to develop a tool and techniques to automate the creation of realistic and representative benchmarks from existing web applications. We propose a record-and-replay approach to capture JavaScript sessions which has sufficient fidelity to accurately recreate key characteristics of the original application, and at the same time is sufficiently flexible that a recording produced on one platform can be replayed on a different one. We describe JSBench, a flexible tool for workload capture and benchmark generation, and demonstrate its use in creating eight benchmarks based on popular sites. Using a variety of runtime metrics collected with instrumented versions of Firefox, Internet Explorer, and Safari, we show that workloads created by JSBench match the behavior of the original web applications.

Improving JavaScript performance by deconstructing the type system

“…To evaluate our enhancements, we use the JSBench benchmark suite [21]. JSBench assembles JavaScript code from real websites such as Amazon, Facebook, Google, Twitter, and Yahoo, by recording a series of user interactions on individual webpages.…”

“…Our experimental results show that a monomorphic inline cache hit requires about 10 instructions Figure 4 compares the number of instructions executed by several JavaScript benchmarks with different V8 optimization levels. We show data for the JSBench [21] Looking at the mean bars, we see that both the optimizing compiler and inline caching substantially benefit the Kraken, Octane, and SunSpider suites. However they have a small or negligible effect on the JSBench suite.…”

“…Figure 5(b) shows a similar example, but this time from actual JavaScript code in the Amazon.com website taken from the JSBench benchmarks [21]. At line 2, object window.amznJQ is constructed using the anonymous function defined between lines 2-7.…”

“…We start by showing that V8 is unable to substantially improve the performance of JavaScript in JSBench [21] -a suite that assembles code from some real websites. Next, we show that a key reason behind this shortfall is the higher degree of type dynamism in website code, which makes it hard for the compiler to predict object types and generate specialized code for them.…”

Interactive record/replay for web application debugging

“…Dolos recordings contain a superset of these inputs; it is possible to synthesize a macro (i.e, automated test case) for use with other tools. The JSBench tool [25] uses this strategy to synthesize standalone web benchmarks. Derived inputs may improve the results of state-exploration tools such as Crawljax [17] by providing real, captured input traces.…”