Moving toward Generalizability? A Scoping Review on Measuring the Impact of Living Labs

Abstract

The living labs (LLs) approach has been applied around the globe to generate innovation within and suited to real-life problems and contexts. Despite the promise of the LL approach for addressing complex challenges like socio-ecological change, there is a gap in practitioner and academic community knowledge surrounding how to measure and evaluate both the performance of a given LL process and its wider impacts. Notably, this gap appears particularly acute in LLs designed to address environmental or agricultural sustainability. This article seeks to verify and address this knowledge gap by conducting an adopted scoping review method which uses a combination of tools for text mining alongside human text analysis. In total, 138 academic articles were screened, out of which 88 articles were read in full and 41 articles were found relevant for this study. The findings reveal limited studies putting forward generalizable approaches or frameworks for evaluating the impact of LLs and even fewer in the agricultural or sustainability sector. The dominant method for evaluation used in the literature is comparative qualitative case studies. This research uncovers a potential tension regarding LL work: the specificity of LL studies works against the development of evaluation indicators and a universal framework to guide the impact assessment of LLs across jurisdictions and studies in order to move toward generalizability.

The Defining Characteristics of Agroecosystem Living Labs

“…For example, the literature offers insights on the way living labs function and foster innovation in terms of living labs as open innovation networks (e.g., [22,23]); the roles of users, stakeholders, and other actors (e.g., [24][25][26]); innovation methodologies and tools (e.g., [27][28][29]); business model innovation [30,31], and many other topics. However the evaluation of the actual impact of a living lab in terms of innovation process, effective adoption, and sustainable changes is still a notable gap [32,33].…”

Living Labs for Public Sector Innovation: insights from a European case study

“…Thus, a concrete challenge of viewing living labs as a public sector innovation method is that the more qualitative aspects, which are the inherent legitimizing factors of living labs, seem to be hard to measure, and hence mainly quantitative performance indicators based on a New Public Management (NPM) logic are typically applied (Ballon et al, 2018;Bronson et al, 2021;Dekker et al, 2021) Moreover, an overall challenge is that the degree of user and/or citizen involvement, which is at the core of living labs, might not be as high as the definition implies. Instead, it seems as if the specificity of living labs is their capacity, and legitimacy, to change the traditional processes of political decision-making processes relative to developing public services (Fuglsang & Hansen, 2022).…”

Living Labs Past Achievements, Current Developments, and Future Trajectories

“…For example, [30,31] document LLs in the context of agriculture and health care, and such articles represent a tradition of firstgeneration LLs, while [27][28][29], focus on co-creation, its process and phases in LLs in secondgeneration LLs. Greve et al [23], Bronson et al [24], Engez et al [25], and Cuomo et al [26] in turn offer various conceptualizations for LLs, which are part of discussion of thirdgeneration LLs. The co-guest editors emphasize that LL generations should not primarily be seen as an advancement indicator, but rather descriptions of larger research traditions that the researchers participate in and belong to.…”

“…Several studies cf. [24,25,30] underline the need for either understanding the sustainability of LLs or the role of LL as a tool or a mechanism for better understanding sustainability both in urban and rural contexts. In this vein, McPhee et al [30] call for future research to future conceptualize agroecosystem LLs and their unique characteristics.…”

“…Studies also emphasize the need for further studies of co-creation processes, their phases, methods, and stakeholders associated processes and phases [27][28][29]31]. Engez et al [24] also propose further research on understanding development of ecosystems and their roles in LLs. Hence, lenses of ecosystems for LLs deserve further attention in LLs [27].…”