Resource Disaggregation Versus Integrated Servers in Data Centers: Impact of Internal Transmission Capacity Limitation

Cheng, Yuxin; Andrade, Marilet De; Wosinska, Lena; Chen, Jiajia

doi:10.1109/ecoc.2018.8535214

Cited by 7 publications

(10 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Related studies demonstrate that emerging high-bandwidth-density silicon-photonic communication technology is a promising means to deliver resource disaggregation with high bandwidth density and low latency [47,58,83]. However, just the photonic link bandwidth and eiciency alone cannot fully satisfy the bandwidth, energy, and latency requirements of fully-lexible, ine-grain, system-wide disaggregation, making system-wide disaggregation impractical in many cases [19,21,55,83]. For this reason, a few studies argue for rack-level disaggregation of GPUs [34,75].…”

Section: Resource Disaggregationmentioning

confidence: 99%

“…Such approaches usually employ a full-system solution where resources can be pooled from across the system. While this approach maximizes the lexibility and range of resource disaggregation, it also increases the overhead to implement resource disaggregation, for instance by requiring long-range communication that stresses bandwidth and increases latency [21,55,83]. As a result, some work focuses on intra-rack disaggregation [34,75].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Case For Intra-rack Resource Disaggregation in HPC

Klenk

Cook

Teh

et al. 2022

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

The expected halt of traditional technology scaling is motivating increased heterogeneity in high performance computing (HPC) systems with the emergence of numerous specialized accelerators. As heterogeneity increases, so does the risk of underutilizing expensive hardware resources if we preserve today’s rigid node configuration and reservation strategies. This has sparked interest in resource disaggregation to enable finer-grain allocation of hardware resources to applications. However, there is currently no data-driven study of what range of disaggregation is appropriate in HPC. To that end, we perform a detailed analysis of key metrics sampled in NERSC’s Cori, a production HPC system that executes a diverse open-science HPC workload. In addition, we profile a variety of deep learning applications to represent an emerging workload. We show that for a rack (cabinet) configuration and applications similar to Cori, a central processing unit (CPU) with intra-rack disaggregation has a 99.5% probability to find all resources it requires inside its rack. In addition, ideal intra-rack resource disaggregation in Cori could reduce memory and NIC resources by 5.36% to 69.01% and still satisfy the worst-case average rack utilization.

show abstract

Section: Resource Disaggregationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Case For Intra-rack Resource Disaggregation in HPC

Klenk

Cook

Teh

et al. 2022

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…IV. PERFORMANCE EVALUATION In this section, we evaluate the performance in terms of resource utilization and VM request blocking probability for different DC architectures using a customized Python-based simulator [15]. CPU-memory resource communications in the fully disaggregated architecture are carried out by the optical interconnect, while the other types of communication between resources can be supported by the electronic switches.…”

Section: Optical Transmission For Communication Between Resourcesmentioning

confidence: 99%

“…Besides average resource utilization of three types of resources and VM request blocking probability, we also show the revenue difference between running VMs in the considered disaggregated scenarios (i.e., partially disaggregated and fully disaggregated) and the integrated server scenario, which is calculated according to the Google's VM pricing system [14], reflecting revenue either gain or loss from the operators' perspective. The VM required resource and request arrival pattern we refer to [15]. Figures 4(a) and 4(b) show the VM request blocking probability and average resource utilization, respectively.…”

Section: Optical Transmission For Communication Between Resourcesmentioning

confidence: 99%

Disaggregated Data Centers: Challenges and Trade-offs

Lin

Cheng²,

Andrade³

et al. 2020

IEEE Commun. Mag.

Self Cite

View full text Add to dashboard Cite

Resource utilization of modern data centers is significantly limited by the mismatch between the diversity of the resources required by running applications and the fixed amount of hardwired resources (e.g., number of central processing unit CPU cores, size of memory) in the server blades. In this regard, the concept of function disaggregation is introduced, where the integrated server blades containing all types of resources are replaced by the resource blades including only one specific function. Therefore, disaggregated data centers can offer high flexibility for resource allocation and hence their resource utilization can be largely improved. In addition, introducing function disaggregation simplifies the system upgrade, allowing for a quick adoption of new generation components in data centers. However, the communication between different resources faces severe problems in terms of latency and transmission bandwidth required. In particular, the CPU-memory interconnects in fully disaggregated data centers require ultra-low latency and ultrahigh transmission bandwidth in order to prevent performance degradation for running applications. Optical fiber communication is a promising technique to offer high capacity and low latency, but it is still very challenging for the state-of-the-art optical transmission technologies to meet the requirements of the fully disaggregated data centers. In this paper, different levels of function disaggregation are investigated. For the fully disaggregated data centers, two architectural options are presented, where optical interconnects are necessary for CPU-memory communications. We review the state-of-the-art optical transmission technologies and carry out performance assessment when employing them to support function disaggregation in data centers. The results reveal that function disaggregation does improve the efficiency of resource usage in the data centers, although the bandwidth provided by the state-of-the-art optical transmission technologies is not always sufficient for the fully disaggregated data centers. It calls for research in optical transmission to fully utilize the advantages of function disaggregation in data centers. I. INTRODUCTION Cloud computing is one of the major services provided by modern data centers (DCs), where users are able to freely choose resources and operating systems (OSs) for running their applications without considering the underlying

show abstract

“…The study on the disaggregation of memory in a DC is still in its early stage. Most studies have focused on the requirements for disaggregated DCs [1][2][3][4][5] with the optical switching-based network architecture [6,10,[17][18][19][20][21][22] and/or rack-scale interconnections [2,[11][12][13][14][15]. These studies demonstrated the feasibility of rack-scale composable systems via building a prototype with peripheral component interconnect express (PCIe) or optical switches.…”

Section: Introductionmentioning

confidence: 99%

Distributed memory access architecture and control for fully disaggregated datacenter network

et al. 2022

View full text Add to dashboard Cite

In this paper, we propose novel disaggregated memory module (dMM) architecture and memory access control schemes to solve the collision and contention problems of memory disaggregation, reducing the average memory access time to less than 1 μs. In the schemes, the distributed scheduler in each dMM determines the order of memory read/write access based on delay‐sensitive priority requests in the disaggregated memory access frame (dMAF). We used the memory‐intensive first (MIF) algorithm and priority‐based MIF (p‐MIF) algorithm that prioritize delay‐sensitive and/or memory‐intensive (MI) traffic over CPU‐intensive (CI) traffic. We evaluated the performance of the proposed schemes through simulation using OPNET and hardware implementation. Our results showed that when the offered load was below 0.7 and the payload of dMAF was 256 bytes, the average round trip time (RTT) was the lowest, ~0.676 μs. The dMM scheduling algorithms, MIF and p‐MIF, achieved delay less than 1 μs for all MI traffic with less than 10% of transmission overhead.

show abstract

Resource Disaggregation Versus Integrated Servers in Data Centers: Impact of Internal Transmission Capacity Limitation

Cited by 7 publications

References 5 publications

A Case For Intra-rack Resource Disaggregation in HPC

A Case For Intra-rack Resource Disaggregation in HPC

Disaggregated Data Centers: Challenges and Trade-offs

Distributed memory access architecture and control for fully disaggregated datacenter network

Contact Info

Product

Resources

About