Lifting the Veil: Drawing insights about design teams from a cognitively-inspired computational model

Abstract

Novel design methodologies are often evaluated through studies involving human designers, but such studies can incur a high personnel cost. It can also be difficult to isolate the effects of specific team or individual characteristics. This work introduces the Cognitively-Inspired Simulated Annealing Teams (CISAT) modeling framework, a platform for efficiently simulating and analyzing human design teams. The framework models a number of empirically demonstrated cognitive phenomena, thus balancing simplicity and direct applicability. This paper discusses the model's composition, and demonstrates its utility through simulating human design teams in a cognitive study. Simulation results are compared directly to the results from human designers. The CISAT model is also used to identify the most beneficial characteristics in the cognitive study. Keywords: computational model, design cognition, teamwork, engineering design Much cognitive research in engineering design has focused on individuals, despite the fact that most engineering design is actually performed by teams (Paulus, Dzindolet, & Kohn, 2011). This work focuses on developing a better understanding of team-based design through computationally simulating the team design process. Empirical studies are a common means for exploring design cognition and for testing new design methodologies. However, these studies can incur a high personnel cost while only returning a limited amount of data. It can also be difficult to isolate the effects of specific characteristics. This work introduces a computational framework that simulates teambased engineering design through creating software agents that directly solve engineering problems. In addition to offering a resource efficient test bed for evaluating design strategies, this framework can be used to test the conclusions from cognitive studies. It can be used to peel apart aspects of human design, and provides a succinct representation of designer behavior. The purpose of the framework is not to replace cognitive studies, but rather to augment traditional methods of investigation, accelerating the discovery of improved design methodologies.A significant amount of work has attempted to simulate the performance of both teams and individuals (Fan & Yen, 2004). For instance, both the Virtual Design Team model, and another model applied to teams at NASA's Jet Propulsion Laboratory, incorporate detailed descriptions of design team organization and interaction (Jin & Levit, 1996;Olson, Cagan, & Kotovsky, 2009). Both models were used to simulate complex design tasks, but were also burdened by high model complexity. For instance, the model by Olson et al. (2009) used approximately 1000 distinct variables, and required nearly 100000 lines of code for implementation. Still other work has utilized agent-based models to explore the formation of mental models during team problem-solving with respect to both interaction structure (Dionne, Sayama, Hao, & James, 2010) and agent memory (Sayama, Farrell, & Dionne, 2011). Mental ...

Agent-Based Modelling and Simulation of Product Development Teams

“…Similarly, in CISAT model [30] eight theory-based characteristics are implemented in agents to adequately represent problem-solving behaviour displayed by the product design team. Namely, agents have a similar goal, interact in irregular intervals, tend to be biased in favour of their designs, focus on most promising alternatives, learn, develop multiple solutions to avoid premature convergence and, by using breadth-and depth-first search strategies, they search solution space until a satisfying solution is found.…”

“…Regarding the modelling of interactions between team members, it can be noticed that all of the models implement collaborative behaviour in the form of either exchange of the current design solutions, e.g. [28,27,31,30], or exchange of relevant information due to task interdependencies, e.g. [23,22].…”

“…For example, in each model where an agent contacts more knowledgeable team member for help, it is implicitly assumed that the agent "knows" the exact level of team members' knowledge. An example of a model where biased, human-like behaviour is explicitly implemented is CISAT [30] where agents have a personal bias when rating design solutions. Further, human mental models constantly change.…”

“…Finally, as it is presumably the biggest limitation of agent-based modelling [3] the validity of listed models needs to be explored. Out of all presented models which can be used for simulation of product development teams, eight models [25,27,31,36,37,38,39,41] reported no verification or validation details, findings of [26,28,34,40] models are reported to be tested for statistical robustness by employing sensitivity analysis, [23,24,35] models are validated by comparison of the model's outcomes with the assumptions made based on the literature or by face validation, and remaining models [13,15,20,30,32,33] are reported to be verified and validated by comparison with the empirical data. However, some of the validated models used the same data for calibration of the model's parameters and validation purposes.…”

Insights from a Latent Semantic Analysis of Patterns in Design Expertise: Implications for Education

“…Based on years of experience solving an assortment of diverse types of problems in different situations, after exploring multiple concepts, expert designers make preliminary and rapid evaluations of a broad range of design options before opting for one of them [18,33]. Eventually they combine both approaches when the design situation demands it [37].…”