An Update to a Functional Modeling Scoring Rubric With Overall and Question-Level Inter-Rater Reliability

A Longitudinal Exploration of Students’ Functional Modeling Abilities

“…In a review of engineering design texts describing functional modeling, Nagel and Bohm identified seven approaches which broadly lead to two types of models: flow-based models and hierarchical tree-based models [17]. Additionally, they proposed their own algorithmic approach [13] and developed a rubric for assessing functional models based on functional modeling conventions for flow-based models [20,21]-the approach their assessment of teaching approaches indicated to be most prevalent in design texts. Toward understanding students' abilities to not only generate functional models, but also, to understand the concept of function, Linsey et al developed the skills test: FunSkill [22].…”

“…Accordingly, a new 12-question rubric was developed in an effort to capture a potential change in this phenomenon as students progressed from sophomore to senior year. This final rubric was based on the Murphy et al rubric for assessing functional models [20] and is provided as in Figure 2. High-level functions concern major system functionality, or the conservation of energy or material.…”

“…For Question 4, where students were asked to draw a functional model, the functional models were scored using an existing functional modeling rubric published in the Journal of Mechanical Design by Murphy et al [20]. The students also independently applied the Murphy et al rubric when scoring Question 4.…”

Board 18: The Impact of Functional Modeling in an Engineering Design Curriculum

“…A summary of research efforts thus far and their related theme(s) is summarized in Table 1. [1][2][3][4][5][6][7][8][9][10][11][12][13] In the 2011 paper surveying functional modeling teaching methods [1] two sub questions were identified to help determine the value functional modeling brings to the design process: 1.) How do you quantitatively assess the quality of any given functional model?…”

“…The initial scoring rubric was successful with relatively high inter-rater reliability [4,5], but also allowed for some interpretation discrepancies [6]. After initial studies using the scaffold approach to teaching and the binary rubric for functional model assessment, additional work sought to assess the impact of teaching function [8,9] as well as rubric refinement [10,12]. Results thus far have indicated that teaching function provides students a mechanism to decompose engineered systems [9,13], and preliminary data suggests the recognition of function improves students' mental models [11].…”