Impact of etcd deployment on Kubernetes, Istio, and application performance

Abstract

Summary This experience article describes lessons learned as we conducted experiments in a Kubernetes‐based environment, the most notable of which was that the performance of both the Kubernetes control plane and the deployed application depends strongly and in unexpected ways on the performance of the etcd database. The article contains (a) detailed descriptions of how networking with and without Istio works in Kubernetes, based on the Flannel Container Networking Interface (CNI) provider in VXLAN mode with IP Virtual Server (IPVS)‐backed Kubernetes Services, (b) a comprehensive discussion about how to conduct load and performance testing using a closed‐loop workload generator, and (c) an open source experiment framework useful for executing experiments in a shared cloud environment and exploring the resulting data. It also shows that statistical analysis may reveal the data resulting from such experiments to be misleading even when careful preparations are made, and that nondeterministic behavior stemming from etcd can affect both the platform as a whole and the deployed application. Finally, it is demonstrated that using high‐performance backing storage for etcd can reduce the occurrence of such nondeterministic behaviors by a statistically significant (P < .05) margin. The implication of this experience article is that systems researchers studying the performance of applications deployed on Kubernetes cannot simply consider their specific application to be under test. Instead, the particularities of the underlying Kubernetes and cloud platform must be taken into account, in particular because their performance can impact that of etcd.

Adaptive and Application-agnostic Caching in Service Meshes for Resilient Cloud Applications

“…For general applicability, we do not explicitly focus on latency or response times. Latency and response times are nonlinear functions of the amount of work that a server has to do [14] and depend on a multitude of factors, such as application code, its deployment, and the underlying hardware resources, the confluence of which causes unexpected behavior in both the application and the control plane [15]. Thus, a more objective and general measurement on algorithm efficiency and performance is to consider the number of requests that are transmitted across the network and, when a specific application is used, the number of bytes such transmissions consist of.…”

MACE: A Mobile Ad-hoc Computing Emulation Framework

“…Furthermore, it is also possible to mix nodes running in different architectures such as x86 and ARM. The same is true when running native applications together with others made for different operating systems such as RiotOS [27], or other embedded OSes that can be compiled as a host application for testing.…”

Is Cloud RAN a Feasible Option for Industrial Communication Network?

“…Fig. 1 is an example given by [17] on how the workload traffic is directed to the service endpoint (the scheduling function) in a Kubernetes node when having a NodePort 3 type of service. Additionally, if several functions are in the chain, which are deployed but hosted by different nodes in a cluster, the workload will be directed over the Virtual Extensible LAN (VXLAN) to the next end-point.…”

“…An example of workload path to reach a service in a typical cloud environment through Kubernetes NodePort deployment. Figure from[17].…”