The role of explainable Artificial Intelligence in high-stakes decision-making systems: a systematic review

Abstract

A high-stakes event is an extreme risk with a low probability of occurring, but severe consequences (e.g., life-threatening conditions or economic collapse). The accompanying lack of information is a source of high-stress pressure and anxiety for emergency medical services authorities. Deciding on the best proactive plan and action in this environment is a complicated process, which calls for intelligent agents to automatically produce knowledge in the manner of human-like intelligence. Research in high-stakes decision-making systems has increasingly focused on eXplainable Artificial Intelligence (XAI), but recent developments in prediction systems give little prominence to explanations based on human-like intelligence. This work investigates XAI based on cause-and-effect interpretations for supporting high-stakes decisions. We review recent applications in the first aid and medical emergency fields based on three perspectives: available data, desirable knowledge, and the use of intelligence. We identify the limitations of recent AI, and discuss the potential of XAI for dealing with such limitations. We propose an architecture for high-stakes decision-making driven by XAI, and highlight likely future trends and directions.

Transferring Black-Box Decision Making to a White-Box Model

“…Similarly, Feature Importance methods, exemplified by techniques like Gradient-based Feature Attribution (Grad-CAM) and Integrated Gradients, provide a granular understanding of input features that influence model outputs [41,42]. These methods complement the presented knowledge-infusion approach by enabling users to understand model reasoning from different perspectives, which can be critical in clinical settings where understanding the why behind a model's decision can be as important as the decision itself [43]. Combining these techniques with the presented approach could potentially yield a more robust framework for XAI in medicine, accommodating a broader range of clinical applications and ensuring that AI-driven decisions are both transparent and trustworthy [44].…”

Explainable Artificial Intelligence in Hydrology: Interpreting Black-Box Snowmelt-Driven Streamflow Predictions in an Arid Andean Basin of North-Central Chile

“…This idea is reinforced when the discipline itself has explicitly recognized the increasingly substantial role of human-water interactions, and where communication of scientific knowledge and the acquisition of feedback from stakeholders are key factors in the progress of the discipline and its mission to be a "science for solutions" [101]. Various authors have begun to analyze and recognize the ethical and accountability implications brought about by the use of AI/ML/DL in hydrological research (see, for example, a recent discussion on the ethical aspects of DL in hydrology [3]), and it is expected that as the future of hydrology is increasingly connected to the provision of solutions to society's water-related problems, the need to generate reliable solutions in a context in which many of them will be generated by AI/ML/DL, especially those linked to high-stakes decision-making, will be more pressing [8,47,89,102].…”

A review of research on RF MEMS for metaverse interactions

“…At the Mobile World Congress 2022, the metaverse made its debut on the congress hot list and became the focus of attention, which was widely noticed and studied. The metaverse itself is not a new technology, but mobile communication technology [1] (5G [2], 6G [3]), artificial intelligence [4] (computer vision [5], decision-making systems, etc [6]), and immersive technology [7,8] (virtual reality (VR), augmented reality (AR), and mixed reality (MR)), cloud computing [9], blockchain [10], and Internet of Things (IoT) human-computer interaction [11] and other existing new generations of information technology comprehensive integration has become a new stage of information technology development, the development of existing technology to a certain stage after the formation of new organizational aggregation methods derived from the new Internet industry. In the prospect of a metaverse-oriented range of applications, the synergistic optimization of hardware solutions must not be neglected, and in hardware systems, there is currently no strong incentive to decentralize metaverse high-level requirements into more detailed sub-system and component-level specifications for comprehensive quantitative benchmarking.…”