Who's the Guinea Pig?

Abstract

A/B/n testing has been adopted by many technology companies as a data-driven approach to product design and optimization. These tests are often run on their websites without explicit consent from users. In this paper, we investigate such online A/B/n tests by using Optimizely as a lens. First, we provide measurement results of 575 websites that use Optimizely drawn from the Alexa Top-1M, and analyze the distributions of their audiences and experiments. Then, we use three case studies to discuss potential ethical pitfalls of such experiments, including involvement of political content, price discrimination, and advertising campaigns. We conclude with a suggestion for greater awareness of ethical concerns inherent in human experimentation and a call for increased transparency among A/B/n test operators. CCS CONCEPTS • Security and privacy → Privacy protections; • Human-centered computing → User studies; Empirical studies in HCI; • Social and professional topics → Codes of ethics;

Automated web content personalization in the automotive sector: The SEAT case

“…Both in academic and industrial literature, online controlled experiments have been widely commented and discussed (Govind, 2017;Kohavi et al, 2009;Lazarova, 2020;Ros and Runeson, 2018;Tang et al, 2010;Xu et al, 2015). However, some authors claim that this method, as traditionally used, is not suitable for the purpose of evaluating personalization (Chen et al, 2019;Das and Ranganath, 2013;Jiang et al, 2019) and propose adapted A/B testing techniques or alternative new methods (Athey and Imbens, 2016;Haupt et al, 2020;Pouget-Abadie et al, 2018;Tu et al, 2021). On the other hand, it is not always possible to run A/B tests as they might be expensive, time-consuming, at times unethical or even not feasible (e.g.…”