Sinan: ML-based and QoS-aware resource management for cloud microservices

Zhang, Yanqi; Hua, Weizhe; Zhou, Zhongkui; Suh, G. Edward; Delimitrou, Christina

doi:10.1145/3445814.3446693

Cited by 127 publications

(48 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the complexity of performance prediction, Seer [38] proposed an online performance prediction system. FIRM [63] and Sinan [82] presented ML-based frameworks for resource sharing across microservices based on isolation techniques for shared resources. They are not applicable to GPU microservices lacking isolation support on GPUs.…”

Section: Related Workmentioning

confidence: 99%

“…FIRM [63] and SINAN [82] minimize the resource usage of traditional CPU microservices while ensuring the required QoS. Since the communication overhead takes a minor part of the latency for CPU microservices (e.g.…”

Section: De Ciencies Of Prior Workmentioning

confidence: 99%

“…In addition, the main memory can be easily scaled to meet the requirements for CPU microservices. FIRM [63] and SINAN [82] are more focused on fast resources (e.g., cores, LLC, memory bandwidth, main memory) reallocation between microservices. There are not such isolation techniques available on GPUs.…”

Section: De Ciencies Of Prior Workmentioning

confidence: 99%

See 2 more Smart Citations

Astraea: towards QoS-aware and resource-efficient multi-stage GPU services

Zhang

Chen

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

Multi-stage user-facing applications on GPUs are widely-used nowadays, and are often implemented to be microservices. Prior research works are not applicable to ensuring QoS of GPU-based microservices due to the di erent communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microservice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to di erent deployments, we propose an auto-scaling GPU communication framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. CCS CONCEPTS• Computer systems organization → Cloud computing; Neural networks; • Networks → Cloud computing.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: De Ciencies Of Prior Workmentioning

confidence: 99%

See 1 more Smart Citation

Astraea: towards QoS-aware and resource-efficient multi-stage GPU services

Zhang

Chen

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

show abstract

“…In 2021, Zhang et al [20] proposed a novel approach composed of a convolutional neural network (CNN) and boosted trees (BT) for dependency and performance analysis of microservices. Their CNN model did not only analyze the inter-dependencies between microservice units for system complexity navigation, but also the time series metrics related to application performance.…”

Section: Evolution Of Machine Learning-based Container Orchestration ...mentioning

confidence: 99%

“…Containers employ a logical packing mechanism that binds both software and dependencies together for application abstraction [6]. Unlike VMs that support hardware-level resource virtualization where each VM has to to suffer from service level objectives (SLO) violations due to higher resource demands and communication costs [20,21]. These factors should all be taken into account during resource provisioning.…”

Section: Introductionmentioning

confidence: 99%

Machine Learning-based Orchestration of Containers: A Taxonomy and Future Directions

Zhong¹,

Xu²,

Rodríguez³

et al. 2021

Preprint

View full text Add to dashboard Cite

Containerization is a lightweight application virtualization technology, providing high environmental consistency, operating system distribution portability, and resource isolation. Existing mainstream cloud service providers have prevalently adopted container technologies in their distributed system infrastructures for automated application management. To handle the automation of deployment, maintenance, autoscaling, and networking of containerized applications, container orchestration is proposed as an essential research problem. However, the highly dynamic and diverse feature of cloud workloads and environments considerably raises the complexity of orchestration mechanisms. Machine learning algorithms are accordingly employed by container orchestration systems for behavior modelling and prediction of multi-dimensional performance metrics. Such insights could further improve the quality of resource provisioning decisions in response to the changing workloads under complex environments. In this paper, we present a comprehensive literature review of existing machine learning-based container orchestration approaches. Detailed taxonomies are proposed to classify the current researches by their common features. Moreover, the evolution of machine learning-based container orchestration technologies from the year 2016 to 2021 has been designed based on objectives and metrics. A comparative analysis of the reviewed techniques is conducted according to the proposed taxonomies, with emphasis on their key characteristics. Finally, various open research challenges and potential future directions are highlighted.

show abstract

A holistic evaluation methodology for configuring production data centers

Wen

Zhang

Cheng

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

Performance evaluation is the basis for choosing appropriate system-level configurations for large-scale data centers. While the change of a system-level configuration would impact lots of jobs in the data centers, traditional load-testing benchmarks are not sufficient to support the decision-making because they cannot accurately reproduce the complex behaviors of a large number of jobs. Therefore, we expect to further evaluate the system configuration based on the production environment. However, there are technical challenges, namely, the lack of a holistic evaluation method that can unite the evaluation results of various jobs, and the uninterruptable production environment that should not be affected by the evaluation procedure. To address these challenges, we propose a holistic performance evaluation methodology and design its implementation platform. We introduce a simple but powerful performance metric, ERU (effectiveness of resource usage), and combine the ERU of involved jobs into a summarized value to measure the effect of a configuration change. We validate our ERU metric by comparing it with the CPI (Cycle per Instruction) and QPS (query per second) metrics, deploy the platform to production data centers and demonstrate the effectiveness for measuring system-level configurations of both software (JVM compiler update) and hardware (NUMA on/off) to save 14.

show abstract

Sinan: ML-based and QoS-aware resource management for cloud microservices

Cited by 127 publications

References 20 publications

Astraea: towards QoS-aware and resource-efficient multi-stage GPU services

Astraea: towards QoS-aware and resource-efficient multi-stage GPU services

Machine Learning-based Orchestration of Containers: A Taxonomy and Future Directions

A holistic evaluation methodology for configuring production data centers

Contact Info

Product

Resources

About