Communicating Algorithmic Process in Online Behavioral Advertising

Eslami, Motahhare; Kumaran, Sneha R. Krishna; Sandvig, Christian; Karahalios, Karrie

doi:10.1145/3173574.3174006

Cited by 115 publications

(92 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In light of this, a recent wave of HCI research has studied what end-users actually desire to understand about ML systems, and how that transparency affects user attitudes and outcomes. Domains of study include recommender systems [4], medicine [12,75], social media [22], creativity [14], and advertisements [23]. While the majority of this work has tended to focus on explaining the reasoning behind specific model decisions, our work instead examines the broader questions that a user may desire to ask of the system as a whole, including components of the ML pipeline that may occur even before a model is built (e.g., data collection, or selection of model design goals).…”

Section: Algorithmic Transparencymentioning

confidence: 99%

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Cai

Winter

Steiner

et al. 2019

Proc. ACM Hum.-Comput. Interact.

335

188

View full text Add to dashboard Cite

Although rapid advances in machine learning have made it increasingly applicable to expert decision-making, the delivery of accurate algorithmic predictions alone is insufficient for effective human-AI collaboration. In this work, we investigate the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant. In a qualitative lab study, we interviewed 21 pathologists before, during, and after being presented deep neural network (DNN) predictions for prostate cancer diagnosis, to learn the types of information that they desired about the AI assistant. Our findings reveal that, far beyond understanding the local, case-specific reasoning behind any model decision, clinicians desired upfront information about basic, global properties of the model, such as its known strengths and limitations, its subjective point-of-view, and its overall design objective-what it's designed to be optimized for. Participants compared these information needs to the collaborative mental models they develop of their medical colleagues when seeking a second opinion: the medical perspectives and standards that those colleagues embody, and the compatibility of those perspectives with their own diagnostic patterns. These findings broaden and enrich discussions surrounding AI transparency for collaborative decision-making, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice. CCS Concepts: • Human-centered computing → Human computer interaction (HCI).

show abstract

Section: Algorithmic Transparencymentioning

confidence: 99%

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Cai

Winter

Steiner

et al. 2019

Proc. ACM Hum.-Comput. Interact.

335

188

View full text Add to dashboard Cite

show abstract

“…A growing body of work characterizes user perceptions of online tracking [30,82,95,101]. Users have a wide variety of reactions to web tracking related to targeted ads.…”

Section: Online Tracking and Targetingmentioning

confidence: 99%

“…Algorithmic processes assign advertisements to users based on inferred profiles. People do not always understand the output of algorithms [29,72] despite having many opinions about what should be done if algorithms are biased [31], imperfect [30], or discriminatory [5,70]. Nonetheless, users can be surprisingly deferential to algorithmic inferences, such as by showing reluctance to make changes to automatically generated profiles [94] or even selfauditing to fit inferences made by an algorithm [86].…”

Section: Online Tracking and Targetingmentioning

confidence: 99%

Oh, the Places You've Been! User Reactions to Longitudinal Transparency About Third-Party Web Tracking and Inferencing

Weinshel

Wei

Mondal

et al. 2019

Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

Internet companies track users' online activity to make inferences about their interests, which are then used to target ads and personalize their web experience. Prior work has shown that existing privacy-protective tools give users only a limited understanding and incomplete picture of online tracking. We present Tracking Transparency, a privacy-preserving browser extension that visualizes examples of long-term, longitudinal information that third-party trackers could have inferred from users' browsing. The extension uses a client-side topic modeling algorithm to categorize pages that users visit and combines this with data about the web trackers encountered over time to create these visualizations. We conduct a longitudinal field study in which 425 participants use one of six variants of our extension for a week. We find that, after using the extension, participants have more accurate perceptions of the extent of tracking and also intend to take privacy-protecting actions. CCS CONCEPTS• Security and privacy → Usability in security and privacy.

show abstract

“…Such complex aggregation already occurs on some websites. For example, Amazon no longer displays the voter average for each product but instead uses a proprietary Machine Learning algorithm to compute the aggregate ratings [15]. The aggregation policy for votes may account for potential biases, say by weighting the votes based on their arrival time (later votes are more susceptible to herding behavior), history of the voter (differentiating novice voters from the more experienced voters), and content type.…”

Section: Informing Policy Designmentioning

confidence: 99%

Quantifying Voter Biases in Online Platforms

Dev

Karahalios

Sundaram

2019

Proc. ACM Hum.-Comput. Interact.

Self Cite

View full text Add to dashboard Cite

In content-based online platforms, use of aggregate user feedback (say, the sum of votes) is commonplace as the "gold standard" for measuring content quality. Use of vote aggregates, however, is at odds with the existing empirical literature, which suggests that voters are susceptible to different biases-reputation (e.g., of the poster), social influence (e.g., votes thus far), and position (e.g., answer position). Our goal is to quantify, in an observational setting, the degree of these biases in online platforms. Specifically, what are the causal effects of different impression signals-such as the reputation of the contributing user, aggregate vote thus far, and position of content-on a participant's vote on content? We adopt an instrumental variable (IV) framework to answer this question. We identify a set of candidate instruments, carefully analyze their validity, and then use the valid instruments to reveal the effects of the impression signals on votes. Our empirical study using log data from Stack Exchange websites shows that the bias estimates from our IV approach differ from the bias estimates from the ordinary least squares (OLS) method. In particular, OLS underestimates reputation bias (1.6-2.2x for gold badges) and position bias (up to 1.9x for the initial position) and overestimates social influence bias (1.8-2.3x for initial votes). The implications of our work include: redesigning user interface to avoid voter biases; making changes to platforms' policy to mitigate voter biases; detecting other forms of biases in online platforms.

show abstract

Communicating Algorithmic Process in Online Behavioral Advertising

Cited by 115 publications

References 27 publications

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

Oh, the Places You've Been! User Reactions to Longitudinal Transparency About Third-Party Web Tracking and Inferencing

Quantifying Voter Biases in Online Platforms

Contact Info

Product

Resources

About