5.4.1 An Experimental Methodology to Evaluate Concept Generation Procedures Based on Quantitative Lifecycle Performance

Abstract

Abstract. This study presents an experimental methodology to measure how concept generation procedures can affect the anticipated lifecycle performance of engineering systems design concepts. The methodology is based on objective and quantitative measurements of anticipated lifecycle performance of the design concepts. It merges cognitive and computer-aided techniques from the fields of collaboration engineering, creativity, and engineering design. It complements the body of existing techniques relying on subjective expert assessments, and other objective metrics not explicitly measuring anticipated lifecycle performance (e.g. development time and cost). Application of the methodology is demonstrated through evaluation of design procedures generating flexibility in engineering systems design. Experiments had ninety participants generate creative design alternatives to a simplified real estate development design problem. Thirty-two teams of two to three participants performed the collaborative design exercise. An online Group-Support System interface enabled efficient data collection and analysis. A computationally efficient mid-fidelity model was used to evaluate flexible design concepts quantitatively based on real options analysis techniques.