Function Modeling: An Analysis of Pause Patterns in Modeling Activities

Abstract

This paper presents a protocol study conducted with mechanical engineering students, where the participants developed a function structure model for a novel design problem. A modeling activity video was recorded for each participant and coded using a protocol analysis. Pauses in the modeling process were analyzed to identify patterns based on pause time and frequency, distribution of pauses over the modeling activity, events following the pauses, and elements added after pauses. Results show that participants used an average of 38% of the modeling time in pauses with a pause frequency of 41%. Moreover, participants were also found to spend more time in pauses during the second and third quarters of the modeling activity. Subsequently, an analysis of pause lengths revealed three different pause groups corresponding to short, intermediate, and long pauses. Participants added elements to the model significantly more frequently, compared to editing and deleting elements. Instances of deleting were found to be more likely to occur after longer pauses, whereas editing was done more frequently after shorter pauses. Participants paused more frequently before adding flows, and more frequent pauses were observed before labeling function compared to adding function blocks. The flows were found to be labeled after pauses infrequently. Finally, limitations of the study are discussed, and future research questions have been identified.

Protocol Analysis in Engineering Design Education Research: Observations, Limitations, and Opportunities

“…Once participants are recruited and a design task is selected, collecting the protocols can begin. Typically, the experiment is set up in a quiet room with minimal distractions (Ball et al, 2004;Carberry et al, 2009, Lammi, 2011Lemons et al, 2010a;Patel et al, 2018;Song & Becker, 2014). This ensures that participants are able to focus on the design task and the recordings are of sufficient quality for accurate transcription.…”

“…In almost all of the reviewed studies, verbalizations were both audio and video recorded, and were sometimes supplemented with additional materials (e.g., screen capturing software, sketches, white board use, etc.) that were created during the session (Adams et al, 2002;Cardella & Tolbert, 2014;Kavakli & Gero, 2002;Lammi, 2011;Patel et al, 2018;Sriram et al, 2015;Williams et al, 2013). Multiple cameras, positioned at various angles, and individual microphones are used to gather the most comprehensive recording of the session (Kannengiesser & Gero, 2017;Lemons et al, 2009;Mentzer et al, 2015;Wells et al, 2016;Williams et al, 2012;Williams et al, 2013).…”

“…As such, PA is not an appropriate tool for routine student assessment or for large populations in a single study (Atman & Bursic, 1998). Researchers may opt to limit the number of participants in their study population, thus reducing the studies' statistical power (Coso et al, 2010;Daly et al, 2018;Moore et al, 2014;Patel et al, 2018). In addition, conclusions from the data become less generalizable, requiring further research efforts to validate the claims.…”

“…Overall, these studies have explored the design cognition of different types of novice designers and showed that PA can be a flexible method that can be adapted to each individual researcher's needs through the application of different coding schemes to protocol data. Recent publications provide evidence that this method continues to be relevant and useful for studying engineering students' design practices (Baily & McFarland, 2018;Dixon & Buckner, 2019;Patel et al, 2018).…”