Engineering problems in machine learning systems

Abstract

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems that employ machine learning and deep learning models, such as automated driving vehicles. In order to use machine learning in a safety-critical system, it is necessary to demonstrate the safety and security of the system through engineering processes. However, thus far, no such widely accepted engineering concepts or frameworks have been established for these systems. The key to using a machine learning model in a deductively engineered system is decomposing the data-driven training of machine learning models into requirement, design, and verification, particularly for machine learning models used in safety-critical systems. Simultaneously, open problems and relevant technical fields are not organized in a manner that enables researchers to select a theme and work on it. In this study, we identify, classify, and explore the open problems in engineering (safety-critical) machine learning systems -that is, in terms of requirement, design, and verification of machine learning models and systems -as well as discuss related works and research directions, using automated driving vehicles as an example. Our results show that machine learning models are characterized by a lack of requirements specification, lack of design specification, lack of interpretability, and lack of robustness. We also perform a gap analysis on a conventional system quality standard SQuARE with the characteristics of machine learning models to study quality models for machine learning systems. We find that a lack of requirements specification and lack of robustness have the greatest impact on conventional quality models.Preprint. Work in progress.

Is Machine Learning Software Just Software: A Maintainability View

“…Finally, as already mentioned as well as pointed out, e.g., by Kuwajima et al [8], pattern-like solutions, such as wrappers, harnesses and workflows, for example, that can be used to embed ML related functions into bigger systems in a more robust fashion form a direction for future software engineering research.…”

Software Engineering for AI-Based Systems: A Survey

“…As an additional facet to above subcategories, 10 of the 17 studies focused specifically on NFRs. While both Horkoff [80] and Kuwajima et al [117] targeted NFRs in general, the other papers were concerned with one or a few specific NFRs, e.g. safety [1,15,120,169] or model performance [12,33,197].…”

A software engineering perspective on engineering machine learning systems: State of the art and challenges

“…In Cyber-Physical systems, a component whose behavior is driven by an ML/DL model obtained via training and updated through a learning process is called a "learning-enabled component" [138]. Recently, in parallel with the success of ML, such systems are referred to as ML systems [141]/applications [140]/solutions [139]. In this paper, I use the term "ML system" as either a software framework, tool, library, or component that provides ML (including DL) functionalities or software systems that include ML components [P56].…”