Efficient orchestration of Node-RED IoT workflows using a Vector Symbolic Architecture

Abstract

Numerous workflow systems span multiple scientific domains and environments, and for the Internet of Things (IoT), Node-RED offers an attractive Web based user interface to execute IoT service-based workflows. However, like most workflow systems, it coordinates the workflow centrally, and cannot run within more transient environments where nodes are mobile. To address this gap, we show how Node-RED workflows can be migrated into a decentralized execution environment for operation on mobile ad-hoc networks, and we demonstrate this by converting a Node-RED based traffic congestion detection workflow to operate in a decentralized environment. The approach uses a Vector Symbolic Architecture (VSA) to dynamically convert Node-Red applications into a compact semantic vector representation that encodes the service interfaces and the workflow in which they are embedded. By extending existing services interfaces, with a simple cognitive layer that can interpret and exchange the vectors, we show how the required services can be dynamically discovered and interconnected into the required workflow in a completely decentralized manner. The resulting system provides a convenient environment where the Node-RED front-end graphical composition tool can be used to orchestrate decentralized workflows. In this paper, we further extend this work by introducing a new dynamic VSA vector compression scheme that compresses vectors for on-the-wire communication, thereby reducing communication bandwidth while maintaining the semantic information content. This algorithm utilizes the holographic properties of the symbolic vectors to perform compression taking into consideration the number of combined vectors along with similarity bounds that determine conflict with other encoded vectors used in the same context. The resulting savings make this approach extremely efficient for discovery in service based decentralized workflows.

Energy efficient 'in memory' computing to enable decentralised service workflow composition in support of multi-domain operations

“…The json file is automatically parsed and converted into binary vectors using the approach described in Reference . 3 The approach uses random binary vectors to describe each letter of the alphabet and any required additional symbols (e.g., punctuation, from which the various keys can be constructed. For example, the key 'company' is constructed from random vectors for each letter of the word as follows:…”

“…In the context of MDO operations, the sensors and services will have been developed and owned by different MDO partners and standard centralized approaches, using formal ontology's to facilitate service definition and service matching for configuring applications are unlikely to work. The use of semantic vector representations to describe the sensors and services offers an alternative paradigm and in [2][3][4] it has been shown that service definition, service matching and decentralized service composition can be achieved using an approach based on Vector Symbolic Architectures (VSAs) and that the approach offers significant advantages in environments where communication is limited or unreliable.…”

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

“…Workflows were represented and communicated as compositional HVs constructed using the primitives for representing sequences (Section 3.3 in [Kleyko et al, 2021c]) and directed acyclic graphs (Section 3.5.2 in [Kleyko et al, 2021c]). In [Simpkin et al, 2020], the approach was implemented in a particular workflow system -Node-RED.…”

Energy efficient spiking neural network neuromorphic processing to enable decentralised service workflow composition in support of multi-domain operations

“…In a previous paper presented at SPIE 2021 11 , a VSA was described that used an hierarchical BSC binding and bundling scheme [12][13][14] . This new approach to workflow orchestration was used to demonstrate how MDO sensor and service descriptions could be represented as HVs and how these HVs could in turn be bound and bundled into higher-level HVs that represent sensor-service workflows.…”