Simulation of Interaction

We evaluate and analyze the quality of click models with respect to their ability to simulate users' click behavior. To this end, we propose distribution-based metrics for measuring the quality of click simulation in addition to metrics that directly compare simulated and real clicks. We perform a comparison of widely-used click models in terms of the quality of click simulation and analyze this quality for queries with di erent frequencies. We nd that click models fail to accurately simulate user clicks, especially when simulating sessions with no clicks and sessions with a click on the rst position. We also nd that click models with higher click prediction performance simulate clicks be er than other models.

show abstract

Reproducible Web Corpora

Kiesel

Kneist

Alshomary

et al. 2018

J. Data and Information Quality

View full text Add to dashboard Cite

The evolution of web pages from static HTML pages toward dynamic pieces of software has rendered archiving them increasingly difficult. Nevertheless, an accurate, reproducible web archive is a necessity to ensure the reproducibility of web-based research. Archiving web pages reproducibly, however, is currently not part of best practices for web corpus construction. As a result, and despite the ongoing efforts of other stakeholders to archive the web, tools for the construction of reproducible web corpora are insufficient or ill-fitted. This article presents a new tool tailored to this purpose. It relies on emulating user interactions with a web page while recording all network traffic. The customizable user interactions can be replayed on demand, while requests sent by the archived page are served with the recorded responses. The tool facilitates reproducible user studies, user simulations, and evaluations of algorithms that rely on extracting data from web pages. To evaluate our tool, we conduct the first systematic assessment of reproduction quality for rendered web pages. Using our tool, we create a corpus of 10,000 web pages carefully sampled from the Common Crawl and manually annotated with regard to reproduction quality via crowdsourcing. Based on this data, we test three approaches to automatic reproduction-quality assessment. An off-the-shelf neural network, trained on visual differences between the web page during archiving and reproduction, matches the manual assessments best. This automatic assessment of reproduction quality allows for immediate bugfixing during archiving and continuous development of our tool as the web continues to evolve.

show abstract

Simulation of Interaction

Cited by 7 publications

References 44 publications

The Pareto Frontier of Utility Models as a Framework for Evaluating Push Notification Systems

The Pareto Frontier of Utility Models as a Framework for Evaluating Push Notification Systems

Evaluating and Analyzing Click Simulation in Web Search

Reproducible Web Corpora

Contact Info

Product

Resources

About