Evaluation in Design-Oriented Research

Abstract

Abstract. Design has been recognized for a long time both as art and as science. In the sixties of the previous century design-oriented research began to draw the attention of scientific researchers and methodologists, not only in technical engineering but also in the social sciences. However, a rather limited methodology for design-oriented research has been developed, especially as to the social sciences. In this article we introduce evaluation methodology and research methodology as a systematic input in the process of designing. A designing cycle is formulated with six stages, and for each of these stages operations, guidelines and criteria for evaluation are defined. All this may be used for a considerable improvement of the process and product of designing.

Business model metrics: an open repository

“…One of the expected implications of the planned expansion was a reduction of headcount by 90 % and major changes in the operations. The analysis was carried out together with the steering committee [consisting of Chief Executive Officer (CEO), Chief Financial Officer (CFO), Chief Operating Officer (COO) and Chief Information Officer (CIO)], resulting in merely 13 h of discussions and brainstorming in three meetings of about 4-5 h. Three of the six stages of the design process were actively explored (Verschuren and Hartog 2005) (see Table 2), primarily making use of CSOFT as a BM approach.…”

A consumer perspective on moving images: The design and adoption of public audiovisual heritage services in the Netherlands

“…This consists of a nominal process for conducting design science research. The research phases cover elements that are proposed in similar design science research processes or methodologies (e.g., Verschuren & Hartog, 2005;Peffers et al, 2008;Offermann et al, 2009;Carlsson et al, 2011). It should be noted that this process is similar to the regulative cycle (Van Strien, 1986;Van Dijk et al, 1991), which is used in social sciences.…”

“…Secondly, the phase includes the acquaintance of the problem domain by the designer or researcher (Verschuren & Hartog, 2005). If the researcher is not up to date and well informed in this respect, the artifact to be designed and produced most probably will not be sufficiently innovative (Csikszentmihalyi, 1996).…”

Mobile business models: organizational and financial design issues that matter

“…Although much has been written about design science from a number of different disciplines, as yet there is no vocabulary for design science, nor are the available theories directly related to design in IS research. Based on an extensive literature study on design research (Hevner et al 2004;Horváth 2004;Verschuren and Hartog 2005), we concluded that, although existing design taxonomies, modelling, philosophies and epistemology, as well as various practical research approaches, can be related to the phases and artefacts of the design process, as yet theories that explain business model performance are to a large extent lacking. Based on our research on business models we came up with the concepts of critical design issue and critical success factor, and specified these concepts for all four business models domains.…”