Descriptive Models of Sequential Decisions in Engineering Design: An Experimental Study

Abstract

Engineering design involves information acquisition decisions such as selecting designs in the design space for testing, selecting information sources, and deciding when to stop design exploration. Existing literature has established normative models for these decisions, but there is lack of knowledge about how human designers make these decisions and which strategies they use. This knowledge is important for accurately modeling design decisions, identifying sources of inefficiencies, and improving the design process. Therefore, the primary objective in this study is to identify models that provide the best description of a designer’s information acquisition decisions when multiple information sources are present and the total budget is limited. We conduct a controlled human subject experiment with two independent variables: the amount of fixed budget and the monetary incentive proportional to the saved budget. By using the experimental observations, we perform Bayesian model comparison on various simple heuristic models and expected utility (EU)-based models. As expected, the subjects’ decisions are better represented by the heuristic models than the EU-based models. While the EU-based models result in better net payoff, the heuristic models used by the subjects generate better design performance. The net payoff using heuristic models is closer to the EU-based models in experimental treatments where the budget is low and there is incentive for saving the budget. This indicates the potential for nudging designers’ decisions toward maximizing the net payoff by setting the fixed budget at low values and providing monetary incentives proportional to saved budget.

A Mathematical Approach to Design Verification Strategies That Incorporate Corrective Activities as Dedicated Decisions

“…One solution to this problem is to first quantify the impact of a designer's domain knowledge and problem framing on the information collection with a descriptive model [20]. Chaudhari et al [21] compared various descriptive models to provide the best description of information acquisition decisions when multiple information sources are present and the total budget is limited. Another solution is to use set-based design to collect information progressively during the process.…”

Design Strategy Network: A deep hierarchical framework to represent generative design strategies in complex action spaces

“…Further, these representations were seen to maintain their characteristics when transferred to design agents or new problems [30], enabling transfer learning. Apart from Bayesian methods, Gaussian processes have also been used to capture human decision-making behavior and have been used for modeling search [31], choosing design parameters, and even making decisions on information acquisition [32,33]. Other recent work has also employed deep learning models to predict human action decisions and capture their behavior.…”

Detecting Bad Design and Bias From Patents

“…In this paper we focus on technical knowledge aimed at supporting the design process, from the conceptual stage down to industrial development (Chaudhari et al 2020). Design knowledge is tacit and embedded in most cases and it is difficult for designers to express their knowledge fully and explicitly.…”