High Reliability in Digital Organizing: Mindlessness, the Frame Problem, and Digital Operations

Abstract

Organizations facing high risks and operating in purely digital domains, such as computer security and many financial services, must meet two, contradictory goals: they need to identify digital threats at scale and speed while also avoiding errors that result from automated processing. Research on high-reliability organizations has identified multiple challenges in reaching these goals simultaneously, because automation often renders organizations' operations "mindless" and unable to cope gracefully with changing, complex situations characteristic of high-risk domains. In digital operations, a special challenge arises from the "frame problem" connected with the inability of algorithms to adapt to environments not identified within their developers' initial cognitive frames. An exploratory, theory-generating case study was conducted within a computer security company (F-Secure) to examine how organizations acting in digital domains may achieve high reliability by mitigating the frame problem. This article examines digital organizing of the epistemic and pragmatic features of operations, along with arrangements of these features that respond to the frame problem. Collective mindfulness is identified as emerging in such a sociotechnical setting via a carefully layered, systemic constellation of (human) mindful and (digital) mindless operations while the organization's core operations remain digital and algorithmic. The findings point to heretofore unexplored reliability challenges associated with digital organizing, alongside several relevant ways to overcome and/or mitigate them.

The impact of IT mindfulness on complex task performance

“…Salovaara et al [18] studied digital high-reliability organisation, which had automated malware detection into an algorithm. When the algorithm could not handle a novel threat, human experts took over since they were capable of mindful actions.…”

“…When the algorithm could not handle a novel threat, human experts took over since they were capable of mindful actions. Routines in the shape of standard operating procedures or algorithms can restrain noticing novel problems, but on the other hand they may allow mindfulness in complex problem areas [18]. Although the task had multiple restrictions, the students had to know how to use a search engine.…”

Artificial Intelligence and Video Game Creation: A Framework for the New Logic of Autonomous Design

“…First, from an operational perspective, human designers delegate a design task to an autonomous tool, set parameters, start the autonomous design tool, and evaluate the outcome and make adjustments to the set of input parameters. Second, considering that problem spaces are evolving and that the same tool might be used for different design situations (and hence problem spaces), autonomous design tools suffer from the frame problem, which describes how algorithms are constrained by the rules (i.e., the knowledge) they currently possess and are hence incapable of reacting to environment states for which they are not prepared (Dennett, 2006;McCarthy & Hayes, 1981;Salovaara, Lyytinen, & Penttinen, 2019). In the case of symbolic approaches, the frame problem would demand that rules are added to the embedded design model to make it applicable to a broader or changing set of design problems (Dennett, 2006).…”

“…However, even if we assume that we can infinitely add rules, such approach will increase the system's complexity and render its performance useless. While connectionist approaches involve learning, they still suffer from the frame problem as they are typically solving "closed-world" problems and remain constrained by the specific goal functions and available data (Salovaara et al, 2019). Therefore, in this section, we turn to the interaction between autonomous design tools and their control units, most notably human designers, in relation to the environment in which they operate.…”

The Role of Information Technology Mindfulness in the Postadoption Stage of Using Personal Health Devices: Cross-Sectional Questionnaire Study in Mobile Health

“…However, most of these studies are conducted in the organizational context and are considered organizational factors and interventions [ 22 ]. Another stream of research focuses on mindfulness in a collective context, such as a group setting, rather than examining mindfulness at the individual level [ 23 ]. Thus, more studies are required to examine the antecedents and resultant effects of IT mindfulness at the individual level in the context of using consumer IT.…”