Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design

Abstract

Many decisions within engineering systems design are typically made by humans. These decisions significantly affect the design outcomes and the resources used within design processes. While decision theory is increasingly being used from a normative standpoint to develop computational methods for engineering design, there is still a significant gap in our understanding of how humans make decisions within the design process. Particularly, there is lack of knowledge about how an individual's domain knowledge and framing of the design problem affect information acquisition decisions. To address this gap, the objective of this paper is to quantify the impact of a designer's domain knowledge and problem framing on their information acquisition decisions and the corresponding design outcomes. The objective is achieved by (i) developing a descriptive model of information acquisition decisions, based on an optimal one-step look ahead sequential strategy, utilizing expected improvement maximization, and (ii) using the model in conjunction with a controlled behavioral experiment. The domain knowledge of an individual is measured in the experiment using a concept inventory, whereas the problem framing is controlled as a treatment variable in the experiment. A design optimization problem is framed in two different ways: a domain-specific track design problem and a domain-independent function optimization problem (FOP). The results indicate that when the problem is framed as a domain-specific design task, the design solutions are better and individuals have a better state of knowledge about the problem, as compared to the domain-independent task. The design solutions are found to be better when individuals have a higher knowledge of the domain and they follow the modeled strategy closely.

Students As Sequential Decision-Makers: Quantifying the Impact of Problem Knowledge and Process Deviation on the Achievement of Their Design Problem Objective

“…The sequential decision-making process described in Section 2.2 has three main activities, namely, information acquisition, information processing, and decision-making. These activities, as illustrated in Figure 3, are modeled by Shergadwala et al [22]. We adopt this model that has been experimentally validated [22].…”

“…These activities, as illustrated in Figure 3, are modeled by Shergadwala et al [22]. We adopt this model that has been experimentally validated [22]. Our focus is on utilizing this model to conduct a simulation study to address the research question discussed in Section 1.…”

“…In effect, the authors [22] model the state of knowledge of an individual about the mathematically unknown objective function f (x) through a Gaussian Process (GP) [23]. An individual's state of knowledge about the constraints g(x) is represented as their beliefs about the constraint feasibility through a logistic regression.…”

“…We choose a 1D design objective function f (x) and constraint function g(x) ≥ 0 to instantiate a design scenario as de-scribed in Section 2.2. While any objective function and constraint inequality could have been chosen, we maintained the consistency of our simulation study with the design problem chosen for the model verification in the study conducted by Shergadwala et al [22]. By fixing the problem we eliminate the effect of problem-specific factors such as problem difficulty.…”

Toward a Theory of Systems Engineering Processes: A Principal–Agent Model of a One-Shot, Shallow Process

“…Contrary to RE and VDD, it is more plausible that the design agents seek to maximize their own objectives. Indeed, there is experimental evidence that the quality of the outcome of a design task is strongly affected by the reward anticipated by the agent [6][7][8]. In other words, the agent decides how much effort and resources to devote to a design task after taking into account the potential reward.…”