Exploring the cost and performance benefits of AWS step functions using a data processing pipeline

Abstract

In traditional cloud computing, dedicated hardware is substituted by dynamically allocated, utility-oriented resources such as virtualized servers. While cloud services are following the pay-as-you-go pricing model, resources are billed based on instance allocation and not on the actual usage, leading the customers to be charged needlessly. In serverless computing, as exemplified by the Functionas-a-Service (FaaS) model where functions are the basic resources, functions are typically not allocated or charged until invoked or triggered. Functions are not applications, however, and to build compelling serverless applications they frequently need to be orchestrated with some kind of application logic. A major issue emerging by the use of orchestration is that it complicates further the already complex billing model used by FaaS providers, which in combination with the lack of granular billing and execution details offered by the providers makes the development and evaluation of serverless applications challenging.Towards shedding some light into this matter, in this work we extensively evaluate the state-of-the-art function orchestrator AWS Step Functions (ASF) with respect to its performance and cost. For this purpose we conduct a series of experiments using a serverless data processing pipeline application developed as both ASF Standard and Express workflows. Our results show that Step Functions using Express workflows are economical when running short-lived tasks with many state transitions. In contrast, Standard workflows are better suited for long-running tasks, offering in addition detailed debugging and logging information. However, even if the behavior of the orchestrated AWS Lambda functions influences both types of workflows, Step Functions realized as Express workflows get impacted the most by the phenomena affecting Lambda functions. CCS Concepts• Software and its engineering → Software performance; Cloud computing.

Task Offloading with Data-Dependent Constraints in Satellite Edge Computing Networks: A Multi-Objective Approach

“…We assumed the SEC network uses AWS pricing [49], which is billed every second. The user cost has three components: (1) cost of using edge servers, (2) cost of transmitting data, and (3) cost of network occupancy.…”

Optimizing AWS lambda code execution time in amazon web services

“…Usually, functions are written in high-level programming languages like Python, Java, or Node.js [25] . NET, and Go [26]. Python is a programming language that is interpreted and has an expressive syntax, which has been likened to executable pseudocode.…”

A probabilistic modeling and evolutionary optimization approach for serverless workflow configuration

“…A compelling serverless application can be built through using a business logic specified in a workflow to orchestrate functions, and hence workflow scheduling is crucial in serverless computing. Based on the function orchestrator AWS Step Functions (ASF), Mathew et al 24 conducted a series of experiments using a serverless data processing pipeline application based on both ASF Standard and Express workflows. Their results revealed that Express workflows are economical for short‐lived tasks whereas Standard workflows are better suited for long‐running tasks.…”