DFaaS: Decentralized Function-as-a-Service for Federated Edge Computing

Ciavotta, Michele; Motterlini, Davide; Savi, Marco; Tundo, Alessandro

doi:10.1109/cloudnet53349.2021.9657141

Cited by 17 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most use Docker and rely on Kubernetes to operate-an assumption that prohibits use on HPC systems that employ batch schedulers. Some systems, such as ChainFaaS [28] and DFaaS [29], support distributed function execution on personal computers and edge nodes. FuncX [6] is the only system that supports remote execution on a federated ecosystem of endpoints in a diverse research cyberinfrastructure spanning from HPC to edge.…”

Section: A Coordinating Remote Executionmentioning

confidence: 99%

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Ward¹,

Pauloski²,

Hayot‐Sasson³

et al. 2023

Preprint

View full text Add to dashboard Cite

Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supercomputers and AI training and inference tasks on specialized accelerators. Here, we present our experiences deploying two AIguided simulation workflows across such heterogeneous systems. A unique aspect of our approach is our use of cloud-hosted management services to manage challenging aspects of crossresource authentication and authorization, function-as-a-service (FaaS) function invocation, and data transfer. We show that these methods can achieve performance parity with systems that rely on direct connection between resources. We achieve parity by integrating the FaaS system and data transfer capabilities with a system that passes data by reference among managers and workers, and a user-configurable steering algorithm to hide data transfer latencies. We anticipate that this ease of use can enable routine use of heterogeneous resources in computational science.

show abstract

Section: A Coordinating Remote Executionmentioning

confidence: 99%

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Ward¹,

Pauloski²,

Hayot‐Sasson³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…NFoS can be adopted in different scenarios. Even though nothing prevents to deploy it in centralized cloud, the most appealing case appears to be its adoption in a federated computing environment at the edge [8] [15], where distributed NFoS platform instances are deployed in resource-constrained edge nodes to serve VNF execution requests from near users.…”

Section: A Involved Actors and Deployment Scenariosmentioning

confidence: 99%

Serverless Computing for NFV: Is it Worth it? A Performance Comparison Analysis

Savi

Banfi

Tundo

et al. 2022

2022 IEEE International Conference on Pervasive Computing and Communications Workshops and Other Affiliated Events (PerCom Work

Self Cite

View full text Add to dashboard Cite

Network Function Virtualization has established itself as one of the most important paradigms towards softwarebased networking. While today Virtual Network Functions (VNFs) are typically deployed in the form of serverful virtualmachine or container-based applications, the emergence of serverless computing opens the door to the possibility of implementing them as serverless functions, with benefits in terms of scalability and resource efficiency. This paper aims to assess whether this really makes sense or not, given the system-level overheads that a serverless computing platform naturally brings.We propose an open source platform designed to optimize the execution of network-intensive VNFs and we implement a data-plane and a control-plane function (i.e., NAT and DHCP responder, respectively) as serverless functions. We carry out extensive benchmarking of performance with their serverful counterparts, implemented as stand-alone containerized applications. Our experience makes it possible to conclude that serverless computing is beneficial for the execution of short-lived and request-based control-plane VNFs, while it should be avoided for the execution of data-plane traffic-intensive VNFs.

show abstract

“…For example, some FaaS systems (e.g., OpenWhisk [4], KNIX [14]) support on-premise deployments on specialized hardware (e.g, GPU), but not on distributed and federated computing resources. Some FaaS systems (e.g., DFaaS [34], ChainFaaS [42]) support function execution in distributed environments, but not on research CI. Here we present funcX, a federated and scalable FaaS platform that enables researchers to decompose applications into functions and execute them on arbitrary remote computers via the FaaS paradigm.…”

Section: Motivations and Requirementsmentioning

confidence: 99%

“…DFaaS [34] is a federated and decentralized FaaS platform for edge computing. It relies on a peer-to-peer network to share the states of edge nodes to balance loads among all the nodes.…”

Section: Related Workmentioning

confidence: 99%

funcX: Federated Function as a Service for Science

Chard

Babuji

et al. 2022

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX's endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX's cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows for transparent, secure, and reliable function execution across the federated ecosystem of endpoints-enabling users to route functions to endpoints based on specific needs. funcX uses containers (e.g., Docker, Singularity, and Shifter) to provide common execution environments across endpoints. funcX implements various container management strategies to execute functions with high performance and efficiency on diverse funcX endpoints. funcX also integrates with an in-memory data store and Globus for managing data that may span endpoints. We motivate the need for funcX, present our prototype design and implementation, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than 130 000 concurrent workers. We show that funcX's container warming-aware routing algorithm can reduce the completion time for 3000 functions by up to 61% compared to a randomized algorithm and the in-memory data store can speed up data transfers by up to 3x compared to a shared file system.

show abstract

DFaaS: Decentralized Function-as-a-Service for Federated Edge Computing

Cited by 17 publications

References 19 publications

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Cloud Services Enable Efficient AI-Guided Simulation Workflows across Heterogeneous Resources

Serverless Computing for NFV: Is it Worth it? A Performance Comparison Analysis

funcX: Federated Function as a Service for Science

Contact Info

Product

Resources

About