The Risks of Machine Learning Systems

Abstract

The speed and scale at which machine learning (ML) systems are deployed are accelerating even as an increasing number of studies highlight their potential for negative impact. There is a clear need for companies and regulators to manage the risk from proposed ML systems before they harm people. To achieve this, private and public sector actors first need to identify the risks posed by a proposed ML system. A system's overall risk is influenced by its direct and indirect effects. However, existing frameworks for ML risk/impact assessment often address an abstract notion of risk or do not concretize this dependence. Keeping discussions of risk at an abstract level puts the onus of defining them on the assessor, which may result in incomplete and inconsistent assessments.We propose to address this gap with a context-sensitive framework for identifying ML system risks comprising two components: a taxonomy of the first-and second-order risks posed by ML systems, and their contributing factors. First-order risks stem from aspects of the ML system, while second-order risks stem from the consequences of first-order risks. These consequences are system failures that result from design and development choices. We explore how different risks may manifest in various types of ML systems, the factors that affect each risk, and how first-order risks may lead to second-order effects when the system interacts with the real world.Throughout the paper, we show how real events and prior research fit into our Machine Learning System Risk framework (MLSR).MLSR operates on ML systems rather than technologies or domains, recognizing that a system's design, implementation, and use case all contribute to its risk. In doing so, it unifies the risks that are commonly discussed in the ethical AI community (e.g., ethical/human rights risks) with system-level risks (e.g., application, design, control risks), paving the way for holistic risk assessments of ML systems.

The Role of Artificial Intelligence Technology in Predictive Risk Assessment for Business Continuity: A Case Study of Greece

“…However, the regression analysis suggests that its impact, while positive, is less pronounced compared to other AI applications. The integration of AI in incident response planning shows a high impact on minimizing business disruptions (Charles et al 2023;Tan et al 2022). This is consistent with OECD's (2022) findings on rising cyber threats and the need for AI-driven incident response systems.…”

“…Additionally, NLP algorithms may be used for social media monitoring, giving businesses the ability to monitor and evaluate public opinion and conversations about their brand or sector (Biolcheva and Valchev 2022;Thekdi and Aven 2023). According to a study by Tan et al (2022), sentiment analysis using Twitter data could predict stock market movements with an accuracy rate of 87.6%. This finding highlights the potential of NLP in recognizing external variables that may have an influence on company continuity.…”

“…The integration of AI in incident response planning has become increasingly critical for minimizing business disruptions during crises, particularly in the realm of cybersecurity (Tan et al 2022). In today's digitally connected world, organizations face a growing number of cyber threats that can jeopardize business continuity.…”