Smart cities aim to make urban life more enjoyable and sustainable but their highly heterogeneous and distributed context creates unique operational challenges. In such an environment, multiple companies work together with government on applications and data streams spanning several management domains. Deploying these applications, each of which consists of several connected services, and maintaining an overview of application topologies remains difficult. Even though cloud modelling languages have been proposed to solve similar issues, they are not well fit for such a heterogeneous environment because they often require an "all or nothing" approach. Moreover, cloud modelling languages add an additional abstraction layer which rarely supports all features of the underlying platform and make it harder to reuse existing knowledge and tools. This research defines service relationships as the key element to modelling applications as topologies of services. We use this definition to pinpoint what is lacking in the state of the art Kubernetes orchestration tools and provide a blueprint for how relationship support can be added to any orchestrator. We present "orcon", a proof of concept orchestrator which extends the Kubernetes API to allow managing relationships between services by adding metadata to service definitions. Our evaluation shows this orchestrator enables lifecycle synchronization and configuration change propagation with an overhead of only 0.44 seconds per service.
Abstract-The data science skills shortage means that those who have the knowledge are under constant pressure to do more with less. While the data science tools are improving at a staggering pace, the operational tools around them can not keep up. Even researchers at Google state that the issue of automatic configuration and dependency management of services is still an "open, hard problem". This manifests itself in data scientists either constantly having to solve operational challenges or having to be in constant close collaboration with a skilled operations team. This paper addresses the operational challenges behind deploying and managing workflows on top of analytics platforms by starting from three key requirements: data scientists want to model their workflows in a reusable way, this model should be automatically deployed, managed and connected to other services, and this solution should be compatible with existing cloud modeling languages, infrastructure, analytics platforms and tools. The paper explores where the state-of-the-art falls short in meeting these requirements, proposes an architecture to solve the open challenges, and implements and evaluates this architecture.
Kubernetes' high resource requirements hamper its adoption in constrained environments such as the edge and fog. Its extensible control plane is a significant contributor to this, consisting of long-lived processes called "controllers" that constantly listen for state changes and use resources even when they are not needed. This paper presents a WebAssembly-based framework for running lightweight controllers on-demand, only when they are needed. This framework extends the WebAssembly System Interface (WASI), in order to run Kubernetes controllers as lightweight Wasm modules. The framework runs these Wasm controllers in a modified version of Wasmtime, the reference WebAssembly (Wasm) runtime, that swaps idle controllers to disk and activates them when needed. A thorough evaluation shows this framework achieves a 64% memory reduction compared to traditional container-based controller frameworks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.