Measuring the Impact of ChatGPT on Fostering Concept Generation in Innovative Product Design

Abstract

The growing demand for innovative and user-centric product design has led to a growing need for effective idea generation methods. In recent years, natural language processing (NLP) tools such as ChatGPT have emerged as a promising solution for supporting idea generation in various domains. This paper investigates a framework for studying the role of ChatGPT in facilitating the ideation process in product design. This investigation measures the impact of ChatGPT on the generation of innovative concepts compared to the use of “classic” design methods. An overview of the state-of-the-art idea generation methods in product design opens the paper. Then, the paper highlights some hypotheses about the impact of ChatGPT on innovative product design, aiming for product augmentation by adding features. The paper then describes the design experience in which ChatGPT is used as a tool for concept generation. Finally, the paper analyzes the dataset, using precise metrics to characterize the participants’ performance and compare them. This analysis allows the paper to argue about the validation/rejection of the hypotheses. The paper concludes with a discussion of the implications of the findings and some suggestions for future research. Along with the paper, the Microsoft Excel workbook used to perform the data analysis is available to the readers to perform their own data collection and analysis. The workbook UX has been carefully studied and developed to make it usable by anyone. At the same time, it should be flexible enough to manage several situations characterized by different numbers of participants, product functions to implement, and generated concepts.

Are generative design tools creative? A characterisation of tools throughout the design process

“…The generative AI design tools capable of operating in the early Concept and Embodiment design stages are general purpose tools built upon natural language processing. The flexibility afforded by the text representation of designs allows the tools to operate across broad design principles with high abstraction and ambiguity (Filippi, 2023). This generality and therefore lack of specialised physics models prevents these tools from operating in the Detail design stage, however.…”

“…This process is illustrated in Figure 2. The 15 tools selected for review were: TRIZ (Gadd, 2011;Petrov, 2023), Functional Analysis (Little, Wood and McAdams, 1997), Brainstorming (Putman and Paulus, 2009), ChatGPT (Filippi, 2023), Morphological Analysis (Arciszewski, 2018), DALL-E (Bing, 2023;Brisco, Hay and Dhami, 2023), CATIA GDE (Dassault Systemes, 2020), Inspire (Lim and Wong, 2018), Ansys (Ansys, 2021), Abaqus (Johnsen, 2013), Sulis (Altair, 2023;Das et al, 2023), Siemens NX (Menninger, Berroth and Jacobs, 2022;Siemens, 2023a), AutoDesk Fusion 360 (AutoDesk, 2023), Simulink (MathWorks, 2023) and Amesim (Siemens, 2023b). The four italicised early-stage design tools in this list are not computational methods.…”

Datasets in design research: needs and challenges and the role of AI and GPT in filling the gaps

“…The paper via several machine design examples, reports hallucinating formulas, selecting the wrong procedures to solve problems, and wrong mathematical calculations. A framework is proposed to assess ChatGPT's impact on generating innovative concepts in product design (Filippi, 2023). The study found that ChatGPT increased the quantity of concepts generated, but concepts were less useful compared to classic design methods.…”

Benchmarking AI design skills: insights from ChatGPT's participation in a prototyping hackathon

“…Recent advancements in, and especially within the last year, artificial intelligence (AI) and natural language processing (NLP) have captured the attention of academia and industry due to the increase in the availability of free, easy-to-use tools such as ChatGPT and similar prompt-based generative AI tools. Multiple engineering disciplines have experimented with and adopted these tools, solving tasks such as calculations in mechanical engineering (Tiro, 2023), assessing engineering students (Qadir, 2023) as a means of fostering concept generation in product design (Filippi, 2023) and writing academic papers (Thunström, 2022). It has already started transforming the way engineers write code (Ahmad et al, 2023), and it has been used as a tool for optimising 3D printing processes (Badini et al, 2023) and to generate functional 3D models iteratively (Nelson et al, 2023).…”