A Replication Scheme for Multiple Fragmentations with Overlapping Fragments

Wiese, Lena; Waage, Tim; Bollwein, Ferdinand

doi:10.1093/comjnl/bxw041

Cited by 7 publications

(7 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, selecting a cluster leader was not practical enough to work in a real-world environment as most DDBSs have the same specification for all of its members (nodes), specifically in the Peer-2-Peer network. On the other hand, the data replication problem (DRP) was deeply addressed by [6] and formulated as an integer linear problem, with an assumption of having overlapping horizontally-divided data. In fact, the replication problem was looked at as an optimization problem to gain the intended aim of having fragments' copies and sites kept at a minimum.…”

Section: Related Workmentioning

confidence: 99%

Towards an Efficient Data Fragmentation, Allocation, and Clustering Approach in a Distributed Environment

Abdalla

Artoli

2019

Information

View full text Add to dashboard Cite

Data fragmentation and allocation has for long proven to be an efficient technique for improving the performance of distributed database systems’ (DDBSs). A crucial feature of any successful DDBS design revolves around placing an intrinsic emphasis on minimizing transmission costs (TC). This work; therefore, focuses on improving distribution performance based on transmission cost minimization. To do so, data fragmentation and allocation techniques are utilized in this work along with investigating several data replication scenarios. Moreover, site clustering is leveraged with the aim of producing a minimum possible number of highly balanced clusters. By doing so, TC is proved to be immensely reduced, as depicted in performance evaluation. DDBS performance is measured using TC objective function. An inclusive evaluation has been made in a simulated environment, and the compared results have demonstrated the superiority and efficacy of the proposed approach on reducing TC.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards an Efficient Data Fragmentation, Allocation, and Clustering Approach in a Distributed Environment

Abdalla

Artoli

2019

Information

View full text Add to dashboard Cite

show abstract

“…For Data Replication, data replication model, which is drawn in ( Sewisy et al, 2017 ) based on ( Wiese et al, 2016 ), is expertly utilized as it has been proven to have huge positive impact on overall DDBS performance. However, this model is slightly modified to have it skilfully complied with proposed work of this paper.…”

Section: Methodsmentioning

confidence: 99%

“…Most of all, replication problem was looked at as an optimization problem to achieve intended goal that fragments’ copies and sites kept minimized. In follow-up work, ( Wiese, 2015 ) was further being extended in ( Wiese et al, 2016 ) and DRP problem was then re-formalized as an integer linear program. Runtime performance was analyzed and data insertion and deletion were addressed as well.…”

Section: Related Workmentioning

confidence: 99%

An optimized approach for simultaneous horizontal data fragmentation and allocation in Distributed Database Systems (DDBSs)

Amer

Sewisy

Elgendy

2017

Heliyon

View full text Add to dashboard Cite

With the substantial ever-upgrading advancement in data and information management field, Distributed Database System (DDBS) is still proven to be the most growingly-demanded tool to handle the accompanied constantly-piled volumes of data. However, the efficiency and adequacy of DDBS is profoundly correlated with the reliability and precision of the process in which DDBS is set to be designed. As for DDBS design, thus, several strategies have been developed, in literature, to be used in purpose of promoting DDBS performance. Off these strategies, data fragmentation, data allocation and replication, and sites clustering are the most immensely-used efficacious techniques that otherwise DDBS design and rendering would be prohibitively expensive. On one hand, an accurate well-architected data fragmentation and allocation is bound to incredibly increase data locality and promote the overall DDBS throughputs. On the other hand, finding a practical sites clustering process is set to contribute remarkably in reducing the overall Transmission Costs (TC). Consequently, consolidating all these strategies into one single work is going to undoubtedly satisfy a massive growth in DDBS influence. In this paper, therefore, an optimized heuristic horizontal fragmentation and allocation approach is meticulously developed. All the drawn-above strategies are elegantly combined into a single effective approach so as to an influential solution for DDBS productivity promotion is set to be markedly fulfilled. Most importantly, an internal and external evaluations are extensively illustrated. Obviously, findings of conducted experiments have maximally been recorded to be in favor of DDBS performance betterment.

show abstract

“…Additionally, we may even envision new and sophisticated data placement strategies for data chunks, beyond the HDFS strategy of sharding chunks across the physical nodes such as to evenly distribute the workload. In general, devising dynamic data placement strategies to improve system performance already is a current field of research [28]- [32]. With data skewness in mind, chunks might also be placed such that due to co-location of complementary chunks, the probability of a physical node working with skewed data is reduced.…”

Section: B Provenance For ML Expertsmentioning

confidence: 99%

The Best of Both Worlds: Challenges in Linking Provenance and Explainability in Distributed Machine Learning

Scherzinger

Seifert

Wiese

2019

2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)

Self Cite

View full text Add to dashboard Cite

Machine learning experts prefer to think of their input as a single, homogeneous, and consistent data set. However, when analyzing large volumes of data, the entire data set may not be manageable on a single server, but must be stored on a distributed file system instead. Moreover, with the pressing demand to deliver explainable models, the experts may no longer focus on the machine learning algorithms in isolation, but must take into account the distributed nature of the data stored, as well as the impact of any data pre-processing steps upstream in their data analysis pipeline. In this paper, we make the point that even basic transformations during data preparation can impact the model learned, and that this is exacerbated in a distributed setting. We then sketch our vision of end-to-end explainability of the model learned, taking the pre-processing into account. In particular, we point out the potentials of linking the contributions of research on data provenance with the efforts on explainability in machine learning. In doing so, we highlight pitfalls we may experience in a distributed system on the way to generating more holistic explanations for our machine learning models.

show abstract

A Replication Scheme for Multiple Fragmentations with Overlapping Fragments

Cited by 7 publications

References 48 publications

Towards an Efficient Data Fragmentation, Allocation, and Clustering Approach in a Distributed Environment

Towards an Efficient Data Fragmentation, Allocation, and Clustering Approach in a Distributed Environment

An optimized approach for simultaneous horizontal data fragmentation and allocation in Distributed Database Systems (DDBSs)

The Best of Both Worlds: Challenges in Linking Provenance and Explainability in Distributed Machine Learning

Contact Info

Product

Resources

About