Hadoop Extensions for Distributed Computing on Reconfigurable Active SSD Clusters

Kaitoua, Abdulrahman; Hajj, Hazem; Saghir, Mazen A. R.; Artail, Hassan; Akkary, Haitham; Awad, Mariette; Sharafeddin, Mageda; Mershad, Khaleel

doi:10.1145/2608199

Cited by 12 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hadoop (also known as Apache Hadoop) is an open-source framework for distributed storage and distributed processing on huge data sets into computer clusters [812]. For Hadoop, FPGAs' applications include energy-efficient acceleration of big data analytics [809], FPGA-accelerated Hadoop cluster for deep learning computations [813], process streaming data from SSDs (Solid State Drives) using FPGAs [814] and low-power Hadoop cluster [815].…”

Section: Big Datamentioning

confidence: 99%

Field Programmable Gate Array Applications—A Scientometric Review

2019

View full text Add to dashboard Cite

Field Programmable Gate Array (FPGA) is a general purpose programmable logic device that can be configured by a customer after manufacturing to perform from a simple logic gate operations to complex systems on chip or even artificial intelligence systems. Scientific publications related to FPGA started in 1992 and, up to now, we found more than 70,000 documents in the two leading scientific databases (Scopus and Clarivative Web of Science). These publications show the vast range of applications based on FPGAs, from the new mechanism that enables the magnetic suspension system for the kilogram redefinition, to the Mars rovers’ navigation systems. This paper reviews the top FPGAs’ applications by a scientometric analysis in ScientoPy, covering publications related to FPGAs from 1992 to 2018. Here we found the top 150 applications that we divided into the following categories: digital control, communication interfaces, networking, computer security, cryptography techniques, machine learning, digital signal processing, image and video processing, big data, computer algorithms and other applications. Also, we present an evolution and trend analysis of the related applications.

show abstract

Section: Big Datamentioning

confidence: 99%

Field Programmable Gate Array Applications—A Scientometric Review

2019

View full text Add to dashboard Cite

show abstract

“…However, the advantages of SSD, such as stability, high bandwidth, fast response time, and low power consumption can be effectively used in Hadoop system [2]. As such, in order to secure price competitiveness taking advantage of performance, some Hadoop system has been adopting hybrid storage system that makes use of the advantages of SSD and HDD of the large capacity [3]. It is to add SSD to the storage device that consists only of HDD and let SSD play the role of cache unit.…”

Section: Introductionmentioning

confidence: 99%

Design and implement of pre-loading SSD cache data using split file on Hadoop MapReduce

Kim

Jeon

2015

Proceedings of the 2015 Conference on Research in Adaptive and Convergent Systems

View full text Add to dashboard Cite

A storage device that is used mainly in the Hadoop system, HDD, has the slow transfer speed and long latency because of its physical limit. In addition, even though the SSD is the only alternative to traditional HDD, the price of it is too expensive. In order to improve this problem, some distributed file systems use SDD as a cache of HDD. In this paper, in the distributed file system that uses SSD as cache, we propose a mechanism that acquires a block ID that is scheduled to be used or already in use and loads it in SSD cache before the I/O request about the block is made in order to increase the performance of SSD cache during performance of Hadoop MapReduce task. In order to verify the reduction of MapReduce processing time and the improvement of Read I/O performance, which were run based on the proposed mechanism, experiments were carried out via Hadoop Benchmark tool. We could verify through the experiments that the performance of Hadoop MapReduce system was about 11% improved in the system where the proposed mechanism was applied than in the distributed file system where the existing SSD cache was applied. CCS Concepts• Computer systems organization➝Distributed architectures;• Software and its engineering➝Software performance.

show abstract

“…Finally, some works propose extensions to Hadoop with SSDs. For instance, [59] proposes extensions to enable clusters of reconfigurable active SSDs to process streaming data from SSDs using FPGAs. VENU [63] is a proposal for an extension to Hadoop that will use SSDs as a cache for the slower HDDs not for all data, but only for those that are expected to benefit from the use of SSDs.…”

Section: Related Workmentioning

confidence: 99%

“…Recently with the advent of faster Solid State Drives (SSDs) research is emerging to test and possibly to exploit the potential of the new technologically advanced drive [14], [59], [60], [68].…”

Section: Introductionmentioning

confidence: 99%

Spreading dynamics in complex networks with applications in vehicular networks

Basaras¹,

Μπασαράς²

View full text Add to dashboard Cite

Η ανάλυση σύμπλεκτων (complex) δικτύων έχει λάβει τεράστια προσοχή τα τελευταία χρόνια, εξαιτίας της ανάπτυξης των online social δικτύων, και της ύπαρξης τεραστίου όγκου δεδομένων σχετικών με τεχνολογικά (Internet, Web, power grids), βιολογικά δίκτυα (αλληλεπιδράσεων πρωτεϊνών, γονιδίων, εξάπλωση ασθενειών), κ.τ.λ. Η παρούσα διατριβή εξερεύνησε το ζήτημα της διάδοσης “πληροφορίας” (φημών, ιών,…) σε σύμπλεκτα δίκτυα, καθώς και εφαρμογές των σχετικών αποτελεσμάτων σε ad hoc δίκτυα οχημάτων. Ειδικότερα, μελετήθηκε το πρόβλημα της μεγιστοποίησης διάδοσης (influence maximization), και αναπτύχθηκαν state-of-the-art αλγορίθμοι για την εύρεση των πιο ικανών διασκορπιστών πληροφορίας (influential spreaders) σε μονοεπίπεδα και πολυεπίπεδα σύμπλεκτα δίκτυα, καθώς και προσαρμογή αυτών σε πρωτόκολλα δρομολόγησης σε ad hoc δίκτυα οχημάτων (vehicular ad hoc networks). Επιπρόσθετα, μελετήθηκε το δυϊκό πρόβλημα της ελαχιστοποίησης διάδοσης “κακοήθους” (malicious) πληροφορίας σε σύμπλεκτα δίκτυα, για πρώτη φορά σε online fashion, και αναπτύχθηκε αλγόριθμος για την επιλογή των καλύτερων παρακωλυτών διάδοσης κακοήθους πληροφορίας. Κάτω από το πρίσμα των προσφάτων εξελίξεων στα self-driving cars, το προαναφερθέν πρόβλημα μελετήθηκε και για ad hoc δίκτυα οχημάτων, και αναπτύχθηκε μια τεχνική black-listing για όσα οχήματα έχουν περάσει στον έλεγχο κάποιου “adversary”. Τέλος, μελετήθηκε και γενικεύτηκε η ισχύς του “παραδόξου της φιλίας” για μια σειρά από τοπολογικά χαρακτηριστικά των κόμβων ενός δικτύου, και αναπτύχθηκαν εφαρμογές του για την μεγιστοποίηση (ελαχιστοποίηση) της διάδοσης της πληροφορίας στα σύμπλεκτα δίκτυα.

show abstract

Hadoop Extensions for Distributed Computing on Reconfigurable Active SSD Clusters

Cited by 12 publications

References 33 publications

Field Programmable Gate Array Applications—A Scientometric Review

Field Programmable Gate Array Applications—A Scientometric Review

Design and implement of pre-loading SSD cache data using split file on Hadoop MapReduce

Spreading dynamics in complex networks with applications in vehicular networks

Contact Info

Product

Resources

About