Explainable machine learning for public policy: Use cases, gaps, and research directions

Abstract

Explainability is highly desired in machine learning (ML) systems supporting high-stakes policy decisions in areas such as health, criminal justice, education, and employment. While the field of explainable ML has expanded in recent years, much of this work has not taken real-world needs into account. A majority of proposed methods are designed with generic explainability goals without well-defined use cases or intended end users and evaluated on simplified tasks, benchmark problems/datasets, or with proxy users (e.g., Amazon Mechanical Turk). We argue that these simplified evaluation settings do not capture the nuances and complexities of real-world applications. As a result, the applicability and effectiveness of this large body of theoretical and methodological work in real-world applications are unclear. In this work, we take steps toward addressing this gap for the domain of public policy. First, we identify the primary use cases of explainable ML within public policy problems. For each use case, we define the end users of explanations and the specific goals the explanations have to fulfill. Finally, we map existing work in explainable ML to these use cases, identify gaps in established capabilities, and propose research directions to fill those gaps to have a practical societal impact through ML. The contribution is (a) a methodology for explainable ML researchers to identify use cases and develop methods targeted at them and (b) using that methodology for the domain of public policy and giving an example for the researchers on developing explainable ML methods that result in real-world impact.

A systematic review of machine learning models for management, prediction and classification of ARDS

“…Future work also should verify the resulting ML algorithm and is explainability methods with actual physician and clinician as a key component of the research. Although a rigorous validating method was proposed by Amarasinghe et al [78], there are currently few studies that fully utilise this method [78].…”

“…Amarasinghe et al [78] proposed a framework to quantify the effectiveness of explainability method to clinician. This method involves a series of survey on how clinician's opinion changed with and without explainability.…”

Explainable and transparent artificial intelligence for public policymaking

“…Even the very concept of what constitutes a good explanation is still under debate currently. Recent works on explainable ML methods for policy development (Amarasinghe et al, 2023) underline the importance of contextualizing ML explanations and highlight the limitations of existing XAI techniques. Furthermore, they highlight the importance of stakeholder's engagement and the need to prioritize policymakers' requirements rather than relying on technology experts to produce explanations for ML-based policies (Bell et al, 2023).…”

Think about the stakeholders first! Toward an algorithmic transparency playbook for regulatory compliance

“…The existing literature and taxonomies on ADS transparency have identified a number of important stakeholders, which include technologists, policymakers, auditors, regulators, humans-in-the-loop, and those individuals affected by the output of the ADS (Meyers et al, 2007;Amarasinghe et al, 2020;Meske e12-8 Andrew Bell et al et al, 2020). While there is some overlap in how these stakeholders may think about transparency, in general, there is no single approach to designing transparent systems for these disparate stakeholder groups, and each of them has their own goals and purposes for wanting to understand an ADS (Sokol and Flach, 2020).…”