No abstract

Contemporary multiprocessor real-time operating systems, such as VxWorks, LynxOS, QNX, and real-time variants of Linux, allow a process to have an arbitrary processor affinity, that is, a process may be pinned to an arbitrary subset of the processors in the system. Placing such a hard constraint on process migrations can help to improve cache performance of specific multi-threaded applications, achieve isolation among applications, and aid in loadbalancing. However, to date, the lack of schedulability analysis for such systems prevents the use of arbitrary processor affinities in predictable hard real-time systems. This paper presents the first analysis of multiprocessor scheduling with arbitrary processor affinities from a real-time perspective. It is shown that job-level fixed-priority scheduling with arbitrary processor affinities is strictly more general than global, clustered, and partitioned job-level fixed-priority scheduling combined. Concerning the more general case of joblevel dynamic priorities, it is shown that global and clustered scheduling are equivalent to multiprocessor real-time scheduling with arbitrary processor affinities. The Linux push and pull scheduler is studied as a reference implementation and two approaches for the schedulability analysis of hard real-time tasks with arbitrary processor affinity masks are presented. In the first approach, the scheduling problem is reduced to "global-like" sub-problems to which existing global schedulability tests can be applied. The second approach is specifically based on response-time analysis and models the response-time computation as a linear optimization problem. The latter linear-programming-based approach has better runtime complexity than the former reduction-based approach. Schedulability experiments show the proposed techniques to be effective. This paper is an extended version of a prior ECRTS 2013 paper. The extensions and new contributions are summarized in Section 1.1.

Developers use Machine Learning (ML) platforms to train ML models and then deploy these ML models as web services for inference (prediction). A key challenge for platform providers is to guarantee response-time Service Level Agreements (SLAs) for inference workloads while maximizing resource e ciency. Swayam is a fully distributed autoscaling framework that exploits characteristics of production ML inference workloads to deliver on the dual challenge of resource e ciency and SLA compliance. Our key contributions are (1) model-based autoscaling that takes into account SLAs and ML inference workload characteristics, (2) a distributed protocol that uses partial load information and prediction at frontends to provision new service instances, and (3) a backend self-decommissioning protocol for service instances. We evaluate Swayam on 15 popular services that were hosted on a production ML-as-a-service platform, for the following service-speci c SLAs: for each service, at least 99% of requests must complete within the response-time threshold. Compared to a clairvoyant autoscaler that always satis es the SLAs (i.e., even if there is a burst in the request rates), Swayam decreases resource utilization by up to 27%, while meeting the service-speci c SLAs over 96% of the time during a three hour window. Microsoft Azure's Swayam-based framework was deployed in 2016 and has hosted over 100,000 services.

No abstract