A Taxonomy of Software Engineering Challenges for Machine Learning Systems: An Empirical Investigation

Abstract

Artificial intelligence enabled systems have been an inevitable part of everyday life. However, efficient software engineering principles and processes need to be considered and extended when developing AI-enabled systems. The objective of this study is to identify and classify software engineering challenges that are faced by different companies when developing software-intensive systems that incorporate machine learning components. Using case study approach, we explored the development of machine learning systems from six different companies across various domains and identified main software engineering challenges. The challenges are mapped into a proposed taxonomy that depicts the evolution of use of ML components in software-intensive system in industrial settings. Our study provides insights to software engineering community and research to guide discussions and future research into applied machine learning.

Software engineering challenges for machine learning applications: A literature review

“…Requirement activities on data-oriented works have brought new challenges. Lwakatare et al [15] and Kim et al [40] reported the difficulty of specifying desirable datasets. Furthermore, the needs to preserve the privacy and safety of sensitive datasets and to ensure legal compliance with a new regulation such as the European General Data Protection Regulation may impact research directions in requirements engineering [18,24,60,105] "A software design (the result) describes the software architecture -that is, how software is decomposed and organized into components -and the interfaces between those components.…”

“…The skills needed for these works may go far beyond the scope of traditional software engineering. Developing the skill set for ML applications is the major challenge related to this KA [15,16,27,32,39,40,59]. The other topics related to this KA were identified as follows:…”

“…Given the various challenges in software engineering of ML, we surmise that SE challenges for ML applications cover a similarly wide range of topics. SE challenges for ML applications have been discussed in many papers [15][16][17][18]21], but to our knowledge, no survey paper has clarified the overview of SE challenges for ML applications, that is, what SE challenges have been discussed? and which SE research topics are closely related to each challenge?…”

The Current State of Industrial Practice in Artificial Intelligence Ethics

“…Yet, despite all the talk in the area recently, outside these incidents highlighting failures, we know little of the current state of practice of ethics in AI. Software engineering researchers have recently begun to understand more broadly how artificial intelligence and machine learning are changing the way the software is being developed [13]. Has the public and academic discussion in the area motivated smaller industry players to develop more ethical AI?…”

Going digital: Disruption and transformation in software‐intensive embedded systems ecosystems

“…As recognized in Bosch, this allows companies to move from a one‐dimensional value network, ie, from supplier to product company to customer, to a multi‐dimensional business network with multiple avenues for monetization using not only the products but also data and other assets. In addition, data is the fuel for AI and machine and deep learning models . With massive data sets available, these models are used to train machine learning models, such as deep neural networks, in order to mimic human intelligence and allow companies to shift through massive data sets in order to stay on top of trends, provide answers to highly complex problems, and derive new insights.…”