A Nutritional Label for Rankings

Ke, Yang; Stoyanovich, Julia; Asudeh, Abolfazl; Howe, Bill; Jagadish, H. V.; Miklau, Gerome

doi:10.1145/3183713.3193568

Cited by 86 publications

(50 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• quickly grasp properties of a given method to choose an appropriate approach for a desired use case given the unified design of the Fact Sheets (as with food nutrition labels [56] the users know what to expect and how to navigate them).…”

Section: Target Audiencementioning

confidence: 99%

“…All of these methods revolve around recording details about the data themselves, e.g., the units of features, the data collection process and their intended purpose. Other researchers argued for a similar approach for predictive models: "model cards for model reporting" [36], "nutrition labels for rankings" [56] and "algorithmic impact assessment" forms [42]. Finally, Arnold et al [1] suggested "fact sheets" for ML services to communicate their capabilities, constrains, biases and transparency.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Explainability fact sheets

Sokol

Flach

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

180

View full text Add to dashboard Cite

Explanations in Machine Learning come in many forms, but a consensus regarding their desired properties is yet to emerge. In this paper we introduce a taxonomy and a set of descriptors that can be used to characterise and systematically assess explainable systems along five key dimensions: functional, operational, usability, safety and validation. In order to design a comprehensive and representative taxonomy and associated descriptors we surveyed the eXplainable Artificial Intelligence literature, extracting the criteria and desiderata that other authors have proposed or implicitly used in their research. The survey includes papers introducing new explainability algorithms to see what criteria are used to guide their development and how these algorithms are evaluated, as well as papers proposing such criteria from both computer science and social science perspectives. This novel framework allows to systematically compare and contrast explainability approaches, not just to better understand their capabilities but also to identify discrepancies between their theoretical qualities and properties of their implementations. We developed an operationalisation of the framework in the form of Explainability Fact Sheets, which enable researchers and practitioners alike to quickly grasp capabilities and limitations of a particular explainable method. When used as a Work Sheet, our taxonomy can guide the development of new explainability approaches by aiding in their critical evaluation along the five proposed dimensions.

show abstract

Section: Target Audiencementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Explainability fact sheets

Sokol

Flach

2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

180

View full text Add to dashboard Cite

show abstract

“…The data management research community is well-positioned to contribute to developing new methods for interpretability. These new contributions can naturally build on a rich body of work on data provenance (see Herschel et al (2017) for a recent survey), on recent work on explaining classifiers (Ribeiro et al 2016) and auditing black box models using causal framework (Datta et al 2016), and on automatically generating "nutritional labels" for data and models (Yang et al 2018). We can all agree that algorithmic decision-making should be fair, even if we do not agree on the definition of fairness.…”

Section: Algorithmic and Data Transparencymentioning

confidence: 99%

Transparency, Fairness, Data Protection, Neutrality

Abiteboul¹,

Stoyanovich

2019

J. Data and Information Quality

Self Cite

View full text Add to dashboard Cite

The data revolution continues to transform every sector of science, industry, and government. Due to the incredible impact of data-driven technology on society, we are becoming increasingly aware of the imperative to use data and algorithms responsibly-in accordance with laws and ethical norms. In this article, we discuss three recent regulatory frameworks: the European Union's General Data Protection Regulation (GDPR), the New York City Automated Decisions Systems (ADS) Law, and the Net Neutrality principle, which aim to protect the rights of individuals who are impacted by data collection and analysis. These frameworks are prominent examples of a global trend: Governments are starting to recognize the need to regulate data-driven algorithmic technology. Our goal in this article is to bring these regulatory frameworks to the attention of the data management community and to underscore the technical challenges they raise and that we, as a community, are wellequipped to address. The main takeaway of this article is that legal and ethical norms cannot be incorporated into data-driven systems as an afterthought. Rather, we must think in terms of responsibility by design, viewing it as a systems requirement.

show abstract

“…Green & Chen [25] demonstrate that human decision-making interacts with algorithmic processing in the ultimate disparate impacts of algorithmic systems. Other scholars have pursued empirical case studies [19,50,52], field evaluations [10,65], context-aware uses of datasets [46,63], and questions of institutional access to real-world data [67]. In each case, attending to end-use provides a richer design space and new opportunities for improvements and safeguards.…”

Section: Introductionmentioning

confidence: 99%

Toward situated interventions for algorithmic equity

Katell

Young

Dailey

et al. 2020

Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

108

View full text Add to dashboard Cite

Research to date aimed at the fairness, accountability, and transparency of algorithmic systems has largely focused on topics such as identifying failures of current systems and on technical interventions intended to reduce bias in computational processes. Researchers have given less attention to methods that account for the social and political contexts of specific, situated technical systems at their points of use. Co-developing algorithmic accountability interventions in communities supports outcomes that are more likely to address problems in their situated context and re-center power with those most disparately affected by the harms of algorithmic systems. In this paper we report on our experiences using participatory and co-design methods for algorithmic accountability in a project called the Algorithmic Equity Toolkit. The main insights we gleaned from our experiences were: (i) many meaningful interventions toward equitable algorithmic systems are non-technical; (ii) community organizations derive the most value from localized materials as opposed to what is "scalable" beyond a particular policy context; (iii) framing harms around algorithmic bias suggests that more accurate data is the solution, at the risk of missing deeper questions about whether some technologies should be used at all. More broadly, we found that community-based methods are important inroads to addressing algorithmic harms in their situated contexts.

show abstract

A Nutritional Label for Rankings

Cited by 86 publications

References 9 publications

Explainability fact sheets

Explainability fact sheets

Transparency, Fairness, Data Protection, Neutrality

Toward situated interventions for algorithmic equity

Contact Info

Product

Resources

About