Evaluation of collaborative modeling processes for knowledge articulation and alignment

Abstract

The evaluation of modeling processes for years has focused on assessing the outcome of modeling, while the process of modeling itself hardly has been subject of examination. Only in recent years, with rising interest in collaborative conceptual modeling approaches, the process of modeling has gained more attention. Different streams of research have focused on examining the sequence of model manipulations or have considered the collaborative modeling to be a negotiation process and shaped evaluation criteria accordingly. This article proposes to add the perspective of articulation and alignment of the topic of modeling to the potential foci of evaluation. Its contribution is to show, how existing research originating in theories of co-construction of knowledge can be adapted for this purpose and how the adopted perspective is complementary to those already available in related work. A comparative field study applying different evaluation methods on a single real world showcase is presented to show the practical applicability of the approach and allows to identify potential for the combination of previously unrelated analytical dimensions.

Elicitation Requirements

“…As such, the concept mapping approach is considered to be suitable for externalization of mental models (Pirnay-Dummer 2006). In the course of mapping, constructs are arranged according to an issue of interest, for example, individual organization of work (Oppl 2006). The constructs are named and structured by associating them.…”

“…These constraints are necessary for further processing, but appear to hinder the modeling process itself (Jørgensen 2004). Most modeling languages force modelers to use representational schemes that do not necessarily correspond to their individual understanding of work (Oppl 2018). This mismatch often leads to situations where the modeling language is inappropriate to express what people consider relevant-"Indeed, I would go so far as to claim that constraining practitioners during early design to use some fixed notation with a fixed semantics would slow them down, by forcing them to pay more attention to the limitations of the notation than to the details of their problem" (Goguen 1993).…”

“…For support of explicit Articulation Work, it has to be assured that all aspects of work considered relevant by people can be expressed by the modeling language (Oppl 2016). Moreover, modeling requires the recognition of relevant real-world phenomena, to abstract and conceptualize them, and to represent them with the means of the modeling language.…”

“…These are non-trivial tasks, which might be very challenging-if not overstraining-for people inexperienced in modeling (Goguen 1993). Articulation Work, however, has to be performed by everybody involved in the work process (Strauss 1988), also-and especially-frontline workers, who very rarely have experience in modeling (Oppl 2017). Figure 2.10 visualizes the recognition of intangible assets, both, on the level of individual actors, and the collective layer.…”

“…This phenomenon is also termed immersion. Immersion facilitates active participation in processes (rather than consumption of visual information) through manipulation of objects (Oppl 2006).…”