Design of Experiments and machine learning for product innovation: A systematic literature review

Abstract

The recent increase in digitalization of industrial systems has resulted in a boost in data availability in the industrial environment. This has favored the adoption of machine learning (ML) methodologies for the analysis of data, but not all contexts boast data abundance. When data are scarce or costly to collect, Design of Experiments (DOE) can be used to provide an informative dataset for analysis using ML techniques. This article aims to provide a systematic overview of the literature on the joint application of DOE and ML in product innovation (PI) settings. To this end, a systematic literature review (SLR) of two major scientific databases is conducted, retrieving 388 papers, of which 86 are selected for careful analysis. The results of this review delineate the state of the art and identify the main trends in terms of experimental designs and ML algorithms selected for joint application on PI. The gaps, open problems, and research opportunities are identified, and directions for future research are provided.

Design choice and machine learning model performances

“…In their recent article, Arboretti et al 2 provided a comprehensive overview of the topics of DOE and ML applied in a product innovation (PI) setting. Arboretti et al 2 highlight both the advantages and challenges of the application of ML in PI, discuss the implications of a joint adoption of DOE and ML, and identify the most common experimental designs employed and ML algorithms chosen for analysis.…”

“…In their recent article, Arboretti et al 2 provided a comprehensive overview of the topics of DOE and ML applied in a product innovation (PI) setting. Arboretti et al 2 highlight both the advantages and challenges of the application of ML in PI, discuss the implications of a joint adoption of DOE and ML, and identify the most common experimental designs employed and ML algorithms chosen for analysis. From their work, some clear advantages of the adoption of ML for data analysis appear evident, including the ability of ML to appropriately model data without requiring assumptions on the underlying distribution, the ability to model complex non-linear relationships, and the general capability of providing a better fit to the data, thus ensuring more accurate predictions and the requirement for a less strict design structure, to the point that undesigned data can be effectively modeled.…”

“…From their work, some clear advantages of the adoption of ML for data analysis appear evident, including the ability of ML to appropriately model data without requiring assumptions on the underlying distribution, the ability to model complex non-linear relationships, and the general capability of providing a better fit to the data, thus ensuring more accurate predictions and the requirement for a less strict design structure, to the point that undesigned data can be effectively modeled. Additionally, a proper adoption of a DOE+ML framework can enhance further advantages, including the minimization of the number of trials through the identification of the most informative combinations of the factor levels, the ability to explore experimental regions omitted by DOE alone, the possibility to automate the experimentation procedure, and others 2 .…”

“…However, it has also been shown that there is a lack of studies that try to provide some guidelines on what experimental designs appear to be more appropriate when ML models are selected as a tool for analysis. Instead, in the vast majority of papers that jointly apply DOE and ML, the choice of one specific design appears to be driven by incidental factors such as the analyst's experience or diffusion in one specific area 2 . The aim of this article is to show in an empirical manner what designs are more promising for collection of data that is then modeled through ML techniques.…”

“…As a collateral result, this will also lead to indications being provided on what predictive analytics techniques to take into consideration when analyzing DOE data. It should be pointed out here that only predictive performance of the algorithms in terms of prediction error will be taken into account, and other aspects, such as the ability to quantify uncertainty of predictions and the possibility of investigating causality, are features that should be considered when choosing an algorithm for prediction or optimization 2 . The authors encourage further research devoted to these topics, but this is outside the scope of the present work.…”