Elicited student mental models reveal students’ understanding of a given system as well as their ability to communicate knowledge of that system to others. Understanding how students form and developmental models of systems is critical to the progress of engineering education. In this work, graduate students’ mental models of common household products are measured before and after instruction on functional modeling and functional decomposition. These mental models are measured using previously published, but still developing mental model instruments. The included systems are a hair dryer, clothes dryer, and vacuum cleaner with accompanying scoring rubrics. Results show statistically significant improvements on average mental model rubric scores on all three given systems after the functional modeling intervention. These results suggest that curriculum content on functional modeling and decomposition likely improves students’ mental models of engineering systems and their ability to communicate their knowledge about those systems. As we improve our understanding of how students form, change, and communicate their mental models of engineering systems, educators can shape curriculum to facilitate the skills necessary for comprehensive systems understanding.

No abstract

Mental models are loosely-defined constructs people form to reason and make predictions about their surroundings. These models are an important aspect of systems thinking for engineers, a concept that emphasizes holistic thinking when working with complex systems which is increasingly important in multiple engineering disciplines. Methods to evaluate systems thinking and mental models of systems traditionally rely on questionnaires, or detailed interactive simulations of specific processes. This work presents a method based on functional modules for evaluating student responses to an instrument based on Lawson’s bicycle problem, intended to elicit students’ mental models of two systems. Students were given a simple outline of the two systems, a hair dryer and a car radiator, and were prompted to fill and label the components required for the system to fulfil the functionality described. This was done in two sessions, once before learning functional modeling, and once after, to utilize the method of scoring to evaluate any changes in their mental models due to exposure to functional modeling. The scoring method identifies common functional modules between two systems using Module Heuristics, and then identifies students’ recognition of those modules. This allows a direct comparison of the functional similarity between the two systems identified by the students and can capture a wider variety of correct answers than simply counting the components a student provides.

Functional modeling is a tool used for system abstraction. By divorcing system function from component structure, functional modeling allows designers to more easily identify design opportunities and compartmentalize product functions, which can lead to innovation during the ideation process. In this paper, we examine the reliability of a rubric used to evaluate student-generated functional models by comparing interrater reliabilities on a question-by-question basis from a previous study where an examination of the reliability of each question was not assessed. We then suggest changes to the rubric in order to improve the rubric’s overall interrater reliability as well as its question-by-question interrater reliability. These rubric alterations include clarification of vague language, inclusion of examples and counter examples, and a procedure for handling nonexistent functional components as opposed to incorrect or “nonsensical” functional components. This work is in contribution to the ongoing development of this functional modeling rubric as an education instrument. As functional modeling becomes more widely accepted in the design community and in engineering curricula, it is important to have a validated evaluation metric with which to assess student-generated functional models.

is a mechanical engineering Ph.D. student at the Georgia Institute of Technology conducting research on design theory and engineering education. He received an undergraduate degree in mechanical engineering and a minor in creative writing from the University of South Florida. Alexander is excited to have received an NSF GRFP Fellowship for research in STEM Education and Learning Science. His research has focused on functional modeling and mental models in order to understand how engineering students develop systems thinking skills. He is also a musician and teaches marching percussion (specifically the marimba and vibraphone) to high school students. After completing his graduate degree, he wants to become academic faculty and start a business as a design consultant.