POSUM: A Portfolio Scheduler for MapReduce Workloads

Voinea, Maria A.; Uta, Alexandru; Iosup, Alexandru

doi:10.1109/bigdata.2018.8622215

Cited by 2 publications

(1 citation statement)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Are there open problems? Our latest study [120], on cluster-based big data workloads, indicates portfolio scheduling can make sub-optimal selections when the performance of the policy is difficult to predict. How to alleviate this problem remains open.…”

Section: Design Of Portfolio Schedulersmentioning

confidence: 99%

The AtLarge Vision on the Design of Distributed Systems and Ecosystems

Iosup

Versluis

Trivedi

et al. 2019

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)

Self Cite

View full text Add to dashboard Cite

High-quality designs of distributed systems and services are essential for our digital economy and society. Threatening to slow down the stream of working designs, we identify the mounting pressure of scale and complexity of (eco-)system, of ill-defined and wicked problems, and of unclear processes, methods, and tools. We envision design itself as a core research topic in distributed systems, to understand and improve the science and practice of distributed (eco-)system design. Toward this vision, we propose the ATLARGE design framework, accompanied by a set of 8 core design principles. We also propose 10 key challenges, which we hope the community can address in the following 5 years. In our experience so far, the proposed framework and principles are practical, and lead to pragmatic and innovative designs for large-scale distributed systems. arXiv:1902.05416v1 [cs.DC] 14 Feb 2019 2. Why Focus on MCS Design?We argue in this section for the timely and important need to focus on MCS design. Not only is (good) design needed (Section 2.1), but we identify an increasing need for good design (Section 2.2) and designers (Section 2.3).We also analyze what good design needs to address, that is, complex challenges from system design (Section 2.4) and from MCS design (Section 2.5).3. We anonymize the venue, but consider it relevant because its held year is after 2014, the venue is a conference, and its ranking is A in CORE18 and green in MSAR14. For comparison, ICDCS has these rankings too.4. We anonymize the university, but consider the course relevant because it is large, it took place after 2014, and the university is ranked in the top-150 (in computer science) in both the THE and the QS 2018 World University Rankings (out of nearly 1,000 universities), and in Webometrics of July 18 (out of over 28,000).

show abstract

Section: Design Of Portfolio Schedulersmentioning

confidence: 99%

The AtLarge Vision on the Design of Distributed Systems and Ecosystems

Iosup

Versluis

Trivedi

et al. 2019

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)

Self Cite

View full text Add to dashboard Cite

show abstract

IoT Cloud Computing Middleware for Crowd Monitoring and Evacuation

Gazis

Katsiri

2021

International Journal of Circuits, Systems and Signal Processing

View full text Add to dashboard Cite

Map-Reduce is a programming model and an associated implementation for processing and generating large data sets. This model has a single point of failure: the master, who coordinates the work in a cluster. On the contrary, wireless sensor networks (WSNs) are distributed systems that scale and feature large numbers of small, computationally limited, low-power, unreliable nodes. In this article, we provide a top-down approach explaining the architecture, implementation and rationale of a distributed fault-tolerant IoT middleware. Specifically, this middleware consists of multiple mini-computing devices (Raspberry Pi) connected in a WSN which implement the Map-Reduce algorithm. First, we explain the tools used to develop this system. Second, we focus on the Map-Reduce algorithm implemented to overcome common network connectivity issues, as well as to enhance operation availability and reliability. Lastly, we provide benchmarks for our middleware as a crowd tracking application for a preserved building in Greece (i.e., M. Hatzidakis’ residence). The results of this study show that IoT middleware with low-power and low-cost components are viable solutions for medium-sized cloud computing distributed and parallel computing centres. Potential uses of this middleware apply for monitoring buildings and indoor structures, in addition to crowd tracking to prevent the spread of COVID-19.

show abstract

POSUM: A Portfolio Scheduler for MapReduce Workloads

Cited by 2 publications

References 38 publications

The AtLarge Vision on the Design of Distributed Systems and Ecosystems

The AtLarge Vision on the Design of Distributed Systems and Ecosystems

IoT Cloud Computing Middleware for Crowd Monitoring and Evacuation

Contact Info

Product

Resources

About