Task spreading and shrinking on multiprocessor systems and networks of workstations

Jacob, Joseph; Lee, Sang Yup

doi:10.1109/71.808157

Cited by 6 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to Jacob and Lee (1999) in cases where the communication or computation characteristics change with time the shrinking of the task may improve on the throughput of the system. For example, the computation of count of k-itemsets is computationally more demanding for small k than for larger k. This means that for small k the computation phase would dominate the communication phase while for large k the communication phase would dominate and therefore a smaller number of processors would be used.…”

Section: Static Approachmentioning

confidence: 99%

See 1 more Smart Citation

Parallel mining of association rules using a lattice based approach

Thomas

2007

Proceedings 2007 IEEE SoutheastCon

View full text Add to dashboard Cite

The discovery of interesting patterns from database transactions is one of the major problems in knowledge discovery in database. One such interesting pattern is the association rules extracted from these transactions. The goal of this research was to develop and implement a parallel algorithm for mining association rules. We implemented a parallel algorithm that used a lattice approach for mining association rules. The Dynamic Distributed Rule Mining (DDRM) is a lattice-based algorithm that partitions the lattice into sublattices to be assigned to processors for processing and identification of frequent itemsets. We implemented the DDRM using a dynamic load balancing approach to assign classes to processors for analysis of these classes in order to determine if there are any rules present in them.Parallel algorithms are required for the mining of association rules due to the very large databases used to store the transactions. Some of the previous parallel algorithms are Count Distribution (CD), Data Distribution (DD), Candidate Distribution (CDD), Intelligent Data Distribution (IDD), and Hybrid Distribution (HD). However the costs associated with these algorithms are hash tree construction, hash tree traversal, communication overhead, input/output (I/O) cost and data movement respectively. These algorithms assign tasks to the processors using a static scheduling scheme. The main challenge for a static scheduling scheme is to determine the amount of time that will be needed to process each task. This information can then be used to compute the total time needed to process all the tasks and to divide these tasks among the processors so that an equal amount of tasks are assigned to each processor using processing time as the unit of measurement.Experimental results show that DDRM utilizes the processors efficiently and performed better than the prefix-based and Partition algorithms that use a static approach to assign classes to the processors. The DDRM algorithm scales well and shows good speedup. I would also like to thank all the members

show abstract

Section: Static Approachmentioning

confidence: 99%

“…Using the finest partition may result in a high overhead associated with the assignment of tasks to the processors. The algorithm complexity and time to assign tasks to processors could be reduced with coarse grained partitioning of the problem (Jacob & Lee, 1999).…”

Section: Static Approachmentioning

confidence: 99%

Parallel mining of association rules using a lattice based approach

Thomas

2007

Proceedings 2007 IEEE SoutheastCon

View full text Add to dashboard Cite

show abstract

“…A possibilidade de alterar partições pode fazer com que o mapeamento de tarefas também se altere. Em [5], é apresentado um modelo que suporta partições dinâmicas.…”

Section: Figura 1 Gerência Distribuída De Processadoresunclassified

Implementação de um Gerenciador Distribuído de Processadores para um Agregado Myrinet

Sangoi¹,

Rose²

2002

Anais Do III Workshop Em Sistemas Computacionais De Alto Desempenho (WSCAD 2002)

View full text Add to dashboard Cite

Este artigo descreve três implementações de um modelo de gerência distribuída de processadores para agregados de pequeno e médio porte. Estas implementações são comparadas com um gerenciamento centralizado, sob reais condições de carga. Devido ás alocações distribuídas serem executadas na máquina paralela, em cooperação com os gerentes instalados em cada nó, são analisados resultados de medições de latência e da interferência causada na execução de algumas aplicações paralelas. Os resultados mostram que a alocação distribuída é viável para este tipo de máquina com tecnologias atuais, podendo ser considerada como uma alternativa para ambientes de execução que suportem partições dinâmicas.

show abstract

“…Bestfit submesh allocation strategies attempt to find a free submesh that results in the least fragmentation in the mesh [16][20] [33]. Task migration [6][10] [14] is proposed to dynamically move jobs to another location so as to reduce fragmentation.…”

Section: Introductionmentioning

confidence: 99%

Reducing Fragmentation on Torus-Connected Supercomputers

Tang

Lan

Desai

et al. 2011

2011 IEEE International Parallel &Amp; Distributed Processing Symposium

View full text Add to dashboard Cite

Torus-based networks are prevalent on leadershipclass petascale systems, providing a good balance between network cost and performance. The major disadvantage of this network architecture is its susceptibility to fragmentation. Many studies have attempted to reduce resource fragmentation in this architecture. Although the approaches suggested can make good allocation decisions reducing fragmentation at job start time, none of them considers a job's walltime, which can cause resource fragmentation when neighboring jobs do not complete closely. In this paper, we propose a walltimeaware job allocation strategy, which adjacently packs jobs that finish around the same time, in order to minimize resource fragmentation caused by job length discrepancy. Event-driven simulations using real job traces from a production Blue Gene/P system at Argonne National Laboratory demonstrate that our walltime-aware strategy can effectively reduce system fragmentation and improve overall system performance.

show abstract

Task spreading and shrinking on multiprocessor systems and networks of workstations

Cited by 6 publications

References 21 publications

Parallel mining of association rules using a lattice based approach

Parallel mining of association rules using a lattice based approach

Implementação de um Gerenciador Distribuído de Processadores para um Agregado Myrinet

Reducing Fragmentation on Torus-Connected Supercomputers

Contact Info

Product

Resources

About