Sources of Risk of AI Systems

Abstract

Artificial intelligence can be used to realise new types of protective devices and assistance systems, so their importance for occupational safety and health is continuously increasing. However, established risk mitigation measures in software development are only partially suitable for applications in AI systems, which only create new sources of risk. Risk management for systems that for systems using AI must therefore be adapted to the new problems. This work objects to contribute hereto by identifying relevant sources of risk for AI systems. For this purpose, the differences between AI systems, especially those based on modern machine learning methods, and classical software were analysed, and the current research fields of trustworthy AI were evaluated. On this basis, a taxonomy could be created that provides an overview of various AI-specific sources of risk. These new sources of risk should be taken into account in the overall risk assessment of a system based on AI technologies, examined for their criticality and managed accordingly at an early stage to prevent a later system failure.

Algorithms and the future of work

“…Understanding the risks and benefits of algorithm-enabled workplace systems should be based on a comprehensive risk evaluation. 62 Risks posed by algorithm-enabled systems generally originate in three areas: (1) errors and biases in the input or training data; (2) flaws in the design of the algorithm or mistakes in coding the algorithm into a programming language; and (3) user disregard of an algorithm's limitations or underlying assumptions, leading to an inappropriate application or incorrect interpretation of system outputs or decisions. 63 The increasing complexity of proprietary algorithmsespecially self-learning algorithms which can change their decision logic during operation-make it difficult for designers, manufacturers, and users to gain an operational understanding about how an algorithm works.…”

“…64,65 Lack of algorithmic transparency can be a major impediment to the assessment and control of new occupational safety and health risks. 62 As algorithmic decision-making is increasing in various societal systems, 66 and in worker management systems, advanced sensor technologies, and robotic devices, 47 attention is focused on ways to attain greater algorithm transparency. [67][68][69]…”

“…47,120 When integrated into workplace systems, algorithms can present a unique taxonomy of risks that may not be addressed in an organization's traditional occupational safety and health risk management approaches. 62 New methods are needed to detect biases in input data, find design errors in proprietary algorithms, and ensure that output decisions are logically derivative of the input data. 63 The risks arising from invasive surveillance, algorithmic bias, loss of autonomy and privacy, inaccurate decision outputs, and work intensification should be added to existing risk assessment and management approaches.…”

“…Enthusiasm over the expected benefits of integrating algorithms into workplace equipment, processes, conditions, and human management systems should be tempered by a full awareness and understanding of their risk profile. Understanding the risks and benefits of algorithm‐enabled workplace systems should be based on a comprehensive risk evaluation 62 …”

AIRO: An Ontology for Representing AI Risks Based on the Proposed EU AI Act and ISO Risk Management Standards

“…There is a growing body of literature on discovering types of risk stemming from AI techniques and algorithms. For example, a taxonomy of AI risk sources, proposed in [12], classifies the sources that impact AI trustworthiness into two categories: sources which deal with ethical aspects and the ones that deal with reliability and robustness of the system. The US National Institute of Standards and Technology (NIST) [13] has developed an AI risk management framework which includes a taxonomy of the characteristics that should be taken into account when dealing with risks.…”