Evaluating Participatory Modeling: Developing a Framework for Cross-Case Analysis

Abstract

Participatory modeling is increasingly recognized as an effective way to assist collective decision-making processes in the domain of natural resource management. This article introduces a framework for evaluating projects that have adopted a participatory modeling approach. This evaluation framework--known as the "Protocol of Canberra"--was developed through a collaboration between French and Australian researchers engaged in participatory modeling and evaluation research. The framework seeks to assess the extent to which different participatory modeling initiatives not only modify perceptions among and interactions between participants, but also contribute to collective decision-making. The article discusses the development of the framework and its application to three case-studies, two from Australia and one from the Pacific Island of the Republic of Kiribati. The article concludes with some comments for future use of the framework in a range of participatory modeling contexts.

Learning from collaborative research on sustainably managing fresh water: implications for ethical research–practice engagement

“…The processes involved participant workshop evaluations, participatory observation, participant interviews, and debriefing sessions and interviews with the project team members. Each evaluation had internal and external evaluators so that the data could be triangulated and reflected upon in comparison to other cases (see, for example, Jones et al 2009). Such rigorous coevaluation throws up ethical challenges, including monitoring and mitigating participant discomfort, for example, due to anxiety or self-consciousness that comes through being video or audio recorded.…”

Implementing Participatory Water Management: Recent Advances in Theory, Practice, and Evaluation

“…Other frameworks have been developed to evaluate the quality of the participation process (e.g., Webler 1995, Rowe and Frewer 2000, Schuett et al 2001. Result evaluations of individual cases (e.g., ) but also of long-term and large-scale participation approaches have also been carried out (e.g., for an evaluation of the international governance approach in the North American Great Lakes region, see Klinke 2009), as well as meta evaluations based on multiple case studies (e.g., Beierle and Cayford 2002, Jones et al 2009, Newig and Fritsch 2009, Etienne 2010. Despite this progress, empirical data are still largely inconclusive as to under which conditions participation typically leads to social learning, that is to say "a process of communicative action where multiple actors collectively learn about and develop an understanding of each other's interests, concerns, and preferences through dialog and deliberation" (Muro and Jeffrey 2012).…”

“…Some of these appear to aid the building of consensus for taking water management decisions, e.g., shared vision modeling, or mapping out diversity of points of view and enhancing stakeholder learning, e.g., companion modeling. Recent trends in the field of participatory forms of modeling have headed in a number of directions including determining what role modeling can play in decision aiding activities (Loucks et al 1985, Hatchuel and Molet 1986, Costanza and Ruth 1998, Belton and Stewart 2002, what methods are most adapted to different contexts (Flood and Jackson 1991, Midgley 1997, Mingers 2001, evaluating the effects and outcomes of participatory modeling exercises (Hare et al 2003, Jones et al 2009, Etienne 2010, and determining who organizes, chooses, and influences the modeling process, and what specific roles they play (LunaReyes et al 2006). …”

Understanding the Mechanisms of Collective Decision Making in Ecological Restoration: An Agent-Based Model of Actors and Organizations

“…This literature concerns how to: (1) bring people to the table, for example with the use of participatory modeling (Becu et al 2008, Jones et al 2009); (2) consider a diverse array of knowledge, perspectives, and values (Beratan 2007, Hermans et al 2007, Steyaert et al 2007); and (3) increase stakeholder participation (e.g., Lal et al 2001, Herath and Prato 2006, Castella 2009, Hayati et al 2009). These studies help us understand how to create and increase participation in collective common pool resource management, but they do not illustrate how group structure and participant interactions influence collective decision making within already established conservation organizations.…”