Automatic Facial Feature Extraction for Predicting Designers' Comfort With Engineering Equipment During Prototype Creation

Abstract

Designers frequently utilize engineering equipment to create physical prototypes during the iterative concept generation and prototyping phases of design. Currently, evaluating designers' efficiency during prototype creation is a manual process that either involves observational or survey based approaches. Real-time feedback when using engineering equipment has the potential to enhance designers' efficiency or mitigate potential injuries that may result from incorrect use of equipment. Toward an automated approach to addressing these challenges, the authors of this work test the hypotheses that (i) there exists a difference in designers' comfort levels before and after they use a piece of engineering prototyping equipment and (ii) a machine learning model predicts the level of comfort a designer has while using engineering prototyping equipment with accuracies greater than random chance. It has been shown that the level of comfort that an individual has while completing a task impacts their performance. The authors investigate whether automatic tracking of designers' facial expressions during prototype creation predicts their level of comfort. A study, involving 37 participants using various engineering equipment, is used to validate the approach. The support vector machine (SVM) regression model yielded a range of R squared values from 0.82 to 0.86 for an equipment-specific model. A general model built to predict comfort level across all engineering equipment yielded an R squared value of 0.68. This work has the potential to transform the manner in which design teams utilize engineering equipment toward more efficient concept generation and prototype creation processes.

Review of the use of neurophysiological and biometric measures in experimental design research

“…Bezawada et al . (2017). If the whole set of technologies capable of extracting inadvertent people's reactions had to be considered, behavioral tools should be included in addition to biometric devices.…”

“…These include, among others, the analysis of hand gestures, which complements biometric measurements in some of the described experiments, and Facial Expression Recognition, whose use is surfacing in design research as well, e.g. Bezawada et al (2017). If the whole set of technologies capable of extracting inadvertent people's reactions had to be considered, behavioral tools should be included in addition to biometric devices.…”

Bounded Kalman filter method for motion-robust, non-contact heart rate estimation

“…It also has the potential of being employed in hospitals to predict the heart rate as patients walk in, without the constraint of having to be in a stationary position. Moreover, it can be combined with Microsoft Kinect to be used as a real time ergonomic system as done by [64] or predicting designers' comfort with engineering equipment [65]. Future research expansions could be devoted to predicting pulse-rate in environments where subjects are made to walk longer distances, gender difference is considered an experimental design factor and activities other than motions performed before the subject's laptops/desktop computers are devised as part of the experimental design.…”

Recognition of Design Fixation via Body Language Using Computer Vision

“…Some scholars have paid attention to design thinking research based on body language. Bezawada et al [31] proposed a machine learning model of automatic facial feature extraction for predicting designers' comfort with engineering equipment during prototype creation.…”