Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach

Abstract

Making internet-of-things (IoT)-based applications context-aware demands large amounts of raw data to be collected, interpreted, stored, and reused or repurposed if needed from many domains and applications. Context is transient but interpreted data can be distinguished from IoT data in many aspects. Managing context in cache is a novel area of research that has been given very little attention. Performance metric-driven adaptive context caching (ACOCA) can have a profound impact on the performance and cost efficiency of context-management platforms (CMPs) when responding to context queries in realtime. Our paper proposes an ACOCA mechanism to maximize both the cost and performance efficiency of a CMP in near realtime. Our novel mechanism encompasses the entire context-management life cycle. This, in turn, distinctively addresses the problems of efficiently selecting context for caching and managing the additional costs of context management in the cache. We demonstrate that our mechanism results in long-term efficiencies for the CMP that have not been observed in any previous study. The mechanism employs a novel, scalable, and selective context-caching agent implemented using the twin delayed deep deterministic policy gradient method. It further incorporates an adaptive context-refresh switching policy, a time-aware eviction policy, and a latent caching decision management policy. We point out in our findings that the additional complexity of adaptation introduced to the CMP through ACOCA is significantly justified, considering the cost and performance gains achieved. Our algorithm is evaluated using a real-world inspired heterogeneous context-query load and a data set based on parking-related traffic in Melbourne, Australia. This paper presents and benchmarks the proposed scheme against traditional and context-aware caching policies. We demonstrate that ACOCA outperforms the benchmarks in both cost and performance efficiency, i.e., up to 68.6%, 84.7%, and 67% more cost efficient compared to traditional data caching policies to cache context, redirector mode, and context-aware adaptive data caching under real-world-like circumstances.

Refresh Rate-Based Caching and Prefetching Strategies for Internet of Things Middleware

“…For a CMP operating in NOD-NOR mode, the core of self-adaptation is the decision about what data should be kept in the cache for serving queries and what data can be retrieved from external context providers in an ad-hoc manner (when the query arrives). Biasness to caching or retrieving in an ad-hoc manner is a function of quality requirements, features of the context query load, features of the context providers, features of the data streams, and context information [15]. Consequently, we can state that efficient proactive context cache management is one of the impending challenges for any context storage including CSMS.…”

“…However, we cannot ignore the fact that the provider set freshness is most reliable until the provider is adequately dependablebased on the consumer's context requirements (e.g., sensors can be unreliable due to wear, or dust/residue accumulation on receptors). Hence, quality-based selection of providers at retrieval time [5] and techniques such as alternate context provider retrievals [15] can be required to maintain the cost and quality of context. The role of context providers in determining the Quality of Context is out of the scope of this paper because, (i) CMPs such as CoaaS rely on cost and quality-based context provider selection mechanisms prior to retrieving IoT data [5], and (ii) assuming the providers and state-of-the-art protocols such as CoAP and AMQP are fast and reliable.…”

“…The loss is defined as the sum of several costs: (i) the price of remote services being called, (ii) the amount of cloud service resources (processing, storage, network) being used by the platform, (iii) the amount of penalty which is returned to the account of the consumer when a query is not serviced in a specified time. Examples can be found in [15,22]. Moreover, the administration of the platform can add any other constraints (e.g., maximum percentage of failed queries for a particular consumer).…”

“…All these factors lead to a high possibility of a context provider becoming unavailable during a certain time-period due to network bottlenecks or handover issues. The unavailability of a data provider, in the case of having no valid data in the cache, will cause CoaaS to either exclude the data from a particular provider from the result set or increase the overall time of serving the query while trying to establish a connection or at least select and retrieve from an alternative context provider in the end (which requires invalidating existing cached context and re-inferencing using the new data where necessary [15]). All these consequences are not acceptable especially if high QoS is needed.…”

IoT-Enabled Few-Shot Image Generation for Power Scene Defect Detection Based on Self-Attention and Global–Local Fusion

“…IoT devices [ 6 , 7 ], such as sensors and smart meters, can be deployed in power systems to gather real-time information on various parameters, such as voltage, current, temperature, and other relevant factors [ 8 , 9 , 10 ]. These IoT devices continuously collect data from the power scene, providing valuable insights into the operation and performance of the power equipment.…”