A configurable rule based classful token bucket filter network request scheduler for the lustre file system

Qian, Yingjin; Li, Xi; Ihara, Shuichi; Zeng, Lingfang; Kaiser, Jürgen; Süß, Tim; Brinkmann, André

doi:10.1145/3126908.3126932

Cited by 30 publications

(18 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While some solutions already encapsulate existing storage systems as reusable building blocks [80,83,95], there is no SDS system that leverages from existing storage subsystems (e.g., QoS provisioning, I/O scheduling) to be re-purposed as programmable storage objects and reused in different storage contexts throughout the I/O path. Such a design would open research opportunities towards programmable storage stacks, and foster reutilization of complementary works [47,65,81,118] and existing storage subsystems [25,26,66].…”

Section: Lessons Learned and Future Directionsmentioning

confidence: 99%

A Survey and Classification of Software-Defined Storage Systems

et al. 2020

View full text Add to dashboard Cite

The exponential growth of digital information is imposing increasing scale and efficiency demands on modern storage infrastructures. As infrastructure complexity increases, so does the difficulty in ensuring quality of service, maintainability, and resource fairness, raising unprecedented performance, scalability, and programmability challenges. Software-Defined Storage (SDS) addresses these challenges by cleanly disentangling control and data flows, easing management, and improving control functionality of conventional storage systems. Despite its momentum in the research community, many aspects of the paradigm are still unclear, undefined, and unexplored, leading to misunderstandings that hamper the research and development of novel SDS technologies. In this article, we present an in-depth study of SDS systems, providing a thorough description and categorization of each plane of functionality. Further, we propose a taxonomy and classification of existing SDS solutions according to different criteria. Finally, we provide key insights about the paradigm and discuss potential future research directions for the field.

show abstract

Section: Lessons Learned and Future Directionsmentioning

confidence: 99%

A Survey and Classification of Software-Defined Storage Systems

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Lustre (Schwan 2003) and Ceph (Weil et al 2006) also provide the bandwidth control mechanisms. Qian et al (2017) integrated the QoS function into Lustre based on the token-bucket algorithm which inspired us a lot. But we provide a more complete QoS design on PVFS in our work.…”

Section: Related Workmentioning

confidence: 99%

Software-defined QoS for I/O in exascale computing

et al. 2019

View full text Add to dashboard Cite

Supercomputers' capability is approaching the exascale level, which enables large computing systems to run more jobs concurrently. Since modern data-intensive scientific applications can sometimes produce millions of I/O requests per second, I/O systems always suffer from heavy workloads and impede the overall performance. How to allocate I/O resources and guarantee the QoS (Quality of Service) for each individual application is becoming an increasingly important question. In this paper, we propose SDQoS, a software-defined QoS framework with the token bucket algorithm, aiming to meet the I/O requirements of concurrent applications contending for the I/O resources and improve the overall performance of the I/O systems. Evaluation shows that SDQoS can effectively control the I/O bandwidth within a 5%-10% deviation and improve the performance by 20% in extreme cases.

show abstract

“…One commonly known example of a greedy shaper is the so-called leaky bucket algorithm [35]. We use this shaping algorithm as an example as it is still commonly used in many areas [18,30,36] and adopted into the Specification of Guaranteed Quality of Service [33] by the IETF. A so-called token is added to a collecting bucket with a definable rate of r .…”

Section: Traffic Shapermentioning

confidence: 99%

Code-Inherent Traffic Shaping for Hard Real-Time Systems

Oehlert

Saidi

Falk

2019

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Modern hard real-time systems evolved from isolated single-core architectures to complex multi-core architectures which are often connected in a distributed manner. With the increasing influence of interconnections in hard real-time systems, the access behavior to shared resources of single tasks or cores becomes a crucial factor for the system’s overall worst-case timing properties. Traffic shaping is a powerful technique to decrease contention in a network and deliver guarantees on network streams. In this paper we present a novel approach to automatically integrate a traffic shaping behavior into the code of a program for different traffic shaping profiles while being as least invasive as possible. As this approach is solely depending on modifying programs on a code-level, it does not rely on any additional hardware or operating system-based functions. We show how different traffic shaping profiles can be implemented into programs using a greedy heuristic and an evolutionary algorithm, as well as their influences on the modified programs. It is demonstrated that the presented approaches can be used to decrease worst-case execution times in multi-core systems and lower buffer requirements in distributed systems.

show abstract

A configurable rule based classful token bucket filter network request scheduler for the lustre file system

Cited by 30 publications

References 16 publications

A Survey and Classification of Software-Defined Storage Systems

A Survey and Classification of Software-Defined Storage Systems

Software-defined QoS for I/O in exascale computing

Code-Inherent Traffic Shaping for Hard Real-Time Systems

Contact Info

Product

Resources

About