Comparison of Spark Resource Managers and Distributed File Systems

Salehian, Soulmaz

doi:10.1109/bdcloud-socialcom-sustaincom.2016.88

Cited by 8 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In terms of velocity, spatiotemporal data in most applications are continuous streams of data. As such, they require expensive computational cost for processing [233]. These challenging characteristics of big data cause multiple issues to various STDM and applications.…”

Section: Big Data and Cloud Computingmentioning

confidence: 99%

Spatiotemporal Data Mining: A Survey on Challenges and Open Problems

Hamdi¹,

Shaban²,

Erradi³

et al. 2021

Preprint

View full text Add to dashboard Cite

Spatiotemporal data mining (STDM) discovers useful patterns from the dynamic interplay between space and time. Several available surveys capture STDM advances and report a wealth of important progress in this field. However, STDM challenges and problems are not thoroughly discussed and presented in articles of their own. We attempt to fill this gap by providing a comprehensive literature survey on state-of-the-art advances in STDM. We describe the challenging issues and their causes and open gaps of multiple STDM directions and aspects. Specifically, we investigate the challenging issues in regards to spatiotemporal relationships, interdisciplinarity, discretisation, and data characteristics. Moreover, we discuss the limitations in the literature and open research problems related to spatiotemporal data representations, modelling and visualisation, and comprehensiveness of approaches. We explain issues related to STDM tasks of classification, clustering, hotspot detection, association and pattern mining, outlier detection, visualisation, visual analytics, and computer vision tasks. We also highlight STDM issues related to multiple applications including crime and public safety, traffic and transportation, earth and environment monitoring, epidemiology, social media, and Internet of Things.

show abstract

Section: Big Data and Cloud Computingmentioning

confidence: 99%

Spatiotemporal Data Mining: A Survey on Challenges and Open Problems

Hamdi¹,

Shaban²,

Erradi³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Distributed File System (DFS) is an approach which allows to store expansible varieties of data. The significant aspect of accessing files is a single namespace for the entire system (Salehian & Yan, 2016). The most common distributed storage system implementation with high fault-tolerance are Hadoop DFS, Google Cloud, Amazon Simple Storage Service (S3) and Tachyon, Li, Ghodsi, Zaharia, Shenker, and Stoica (2014).…”

Section: Proposed Systemmentioning

confidence: 99%

Distributed image retrieval with colour and keypoint features

Łągiewka

Korytkowski

Scherer

2019

Journal of Information and Telecommunication

View full text Add to dashboard Cite

Content-based image retrieval poses many problems to computer systems. The content of images has to be described by some feature extraction methods. As image databases are often very large, they are sometimes to complex to be processed by traditional computing methods. We have to use big data solutions to fast retrieve images. The paper presents a system for retrieving images in relational databases in a distributed environment. Content of the query image and images in the database is compared using global colour information and local image keypoints. Image keypoint descriptors are indexed by fuzzy sets directly in a relational database by our algorithm. The process is distributed to several machines thanks to the Apache Hadoop software framework with HDFS.

show abstract

“…Blockchain with its decentralization and security nature has the incredible potential to further develop enormous information administrations and applications [17]. The speed of Hadoop is ideal for batch processing of archive data, on the other hand the performance of Apache Spark is excellent for interactive task and real time analysis [23].…”

Section: Introductionmentioning

confidence: 99%

Domain and Challenges of Big Data and Archaeological Photogrammetry With Blockchain

2022

View full text Add to dashboard Cite

With gigantic growth of data volume that is moved across the web links today, there has been a gigantic measure of perplexing information produced. Extremely huge sets of data including universities, organizations framework, institution gas, petroleum sector, healthcare, that have so enormous thus complex information with more differed structure. The major challenge is how to handle this significant volume of data, also in archaeological photogrammetry which alluded to as Big Data. Although big data has to be securely flying and conveyed through the internet. It cannot be controlled with regular conventional methods that fail to handle it, so there is a need for more up-to-date developed tools. The big data have frequently divided into V's characteristics beginning from three V's: volume, velocity and variety. The initial three V's have been stretched out during time through researches to arrive 56 V's till now. Among them are three newfound by the author that implies it multiplied near twenty times. Researcher had to dive to search for all of these characteristics in many researches to detect and build comparisons to answer the old, current, and restored essential inquiry, "how many V's aspects (characteristics) in big data with archaeological photogrammetry and blockchain". This paper provides a comprehensive overview of all secured big data V's (characteristics) as well as their strength and limitations with archaeological photogrammetry and blockchain.

show abstract

Comparison of Spark Resource Managers and Distributed File Systems

Cited by 8 publications

References 10 publications

Spatiotemporal Data Mining: A Survey on Challenges and Open Problems

Spatiotemporal Data Mining: A Survey on Challenges and Open Problems

Distributed image retrieval with colour and keypoint features

Domain and Challenges of Big Data and Archaeological Photogrammetry With Blockchain

Contact Info

Product

Resources

About