Affinity Scheduling and the Applications on Data Center Scheduling with Data Locality

Kavousi, Mohammadamir

doi:10.48550/arxiv.1705.03125

Cited by 2 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the insights of this paper, for data parallel processing of big data, more sophisticated algorithms based on MapReduce can be used for speeding up the processing time, e.g. look at [36,39,5,40,19].…”

Section: Related Workmentioning

confidence: 99%

Estimating the Rating of the Reviews Based on the Text

Kavousi

Saadatmand

2018

Data Analytics and Learning

Self Cite

View full text Add to dashboard Cite

User-generated texts such as reviews and social media are valuable sources of information. Online reviews are important assets for users to buy a product, see a movie, or make a decision. Therefore, rating of a review is one of the reliable factors for all users to read and trust the reviews. This paper analyzes the texts of the reviews to evaluate and predict the ratings. Moreover, we study the effect of lexical features generated from text as well as sentimental words on the accuracy of rating prediction. Our analysis show that words with high information gain score are more efficient compared to words with high TF-IDF value. In addition, we explore the best number of features for predicting the ratings of the reviews.

show abstract

Section: Related Workmentioning

confidence: 99%

Estimating the Rating of the Reviews Based on the Text

Kavousi

Saadatmand

2018

Data Analytics and Learning

Self Cite

View full text Add to dashboard Cite

show abstract

“…After all map tasks are processed, one or multiple reduce tasks combine the results of all map tasks to produce the final result on the big data-set. Note that processing of big data-sets are either map-intensive or only consist of map task [14], [15], [16], [17], [18], so our focus in this paper is on map task scheduling. Each map task can be processed by any of the servers, but the service rate is faster in the servers that have the data chunk associated with the task, which are referred as local servers.…”

Section: Introductionmentioning

confidence: 99%

The Power of d Choices in Scheduling for Data Centers with Heterogeneous Servers

Moaddeli¹,

Ahmadi²,

Abhar³

2019

Preprint

View full text Add to dashboard Cite

MapReduce framework is the de facto in big data and its applications, in which the big data-set is split into small data chunks and the data chunks are replicated on different servers, three servers by default, among thousands of servers. The heterogeneous server structure of the system, where the pair task-server determines the task processing speed, makes the scheduling much harder than scheduling for systems with homogeneous servers. Throughput optimality of the system on one hand and delay optimality on the other hand creates a dilemma for assigning tasks to servers. Unless many research has been done on both heuristic and theoretical algorithms for scheduling for such a system with multi-level data locality, the optimal scheduling algorithm is still an open problem. The JSQ-MaxWeight and Balanced-Pandas algorithms are the states of the arts algorithms with theoretical guarantees on throughput and delay optimality for systems with two and three levels of data locality. However, the scheduling complexity of these two algorithms are way too much. Hence, we use the power of d choices algorithm combined with the Balanced-Pandas algorithm and the JSQ-MaxWeight algorithm, and compare the complexity of the simple algorithms and the power of d choices versions of them. We will further show that the Balanced-Pandas algorithm combined with the power of the d choices, Balanced-Pandas-Pod, not only performs better than simple Balanced-Pandas, but also is less sensitive to the parameter d than the combination of the JSQ-MaxWeight algorithm and the power of the d choices, JSQ-MaxWeight-Pod. In fact in our extensive simulation results, the Balanced-Pandas-Pod algorithm is performing better than the simple Balanced-Pandas algorithm in low and medium loads, where data centers are usually performing at, and performs almost the same as the Balanced-Pandas algorithm at high loads. The reason is that the combination of the two algorithms enhances the data locality of tasks and prioritize local and rack local service to remote service more than the Balanced-Pandas algorithm. Note that the load balancing complexity of Balanced-Pandas and JSQ-MaxWeight algorithms are O(M ), where M is the number of servers in the system which is in the order of thousands servers, whereas the complexity of Balanced-Pandas-Pod and JSQ-MaxWeight-Pod are O(1), that makes the central scheduler faster and saves energy. Moreover, as mentioned, our proposed Balanced-Pandas-Pod has better performance than the existing algorithms that makes it superior to the state-of-the-art algorithms.

show abstract

Affinity Scheduling and the Applications on Data Center Scheduling with Data Locality

Cited by 2 publications

References 10 publications

Estimating the Rating of the Reviews Based on the Text

Estimating the Rating of the Reviews Based on the Text

The Power of d Choices in Scheduling for Data Centers with Heterogeneous Servers

Contact Info

Product

Resources

About