Performance Evaluations of Distributed File Systems for Scientific Big Data in FUSE Environment

Lee, Jun-Yeong; Kim, Moon-Hyun; Shah, Syed Asif Raza; Ahn, S. U.; Yoon, Heejun; Noh, S. Y.

doi:10.3390/electronics10121471

Cited by 6 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Digital new media art has been the most popular design approach used in the modern exhibition sector as we enter the twenty-rst century. Advanced visual arts technology has modernized the display industry and display systems have evolved from simple to more complex [2].…”

Section: Introductionmentioning

confidence: 99%

Visual Art Design of Digital Works Guided by Big Data

Cheng

2022

Mobile Information Systems

View full text Add to dashboard Cite

With the rapid development of digital technology, the development speed of digital media is also relatively fast. Digital media technology has a great impact on people’s lifestyles and aesthetic concepts, and it also has a greater impact on visual art, creative thinking communication methods, and expression methods. In this study, the quality enhancement of digital images has been intensively studied based on the guidance of big data of eye-movement gaze points. A large amount of visual data are collected from public social resources, and the optimization research of image sensory quality is carried out in-depth using the acquired big data. Next, the region of interest (ROI) is obtained by combining the data with a two-dimensional Gaussian distribution model-fitting method, and the obtained data clustered and improved based on the K-means clustering algorithm to obtain ROI fixation points. Finally, discontinuities in the choice of sharpness in graphics and video playback are pointed out, and the final fixation data analysis is utilized. Results show that targeted optimization is very effective in improving the quality of digital images and saving space, enabling users to enjoy higher-quality visual digital images. The proposed method can be used to improve the dynamic resolution of images and videos.

show abstract

Section: Introductionmentioning

confidence: 99%

Visual Art Design of Digital Works Guided by Big Data

Cheng

2022

Mobile Information Systems

View full text Add to dashboard Cite

show abstract

“…There are various studies that have been conducted on DFSs. Lee et al [17] implemented disk-based DFSs for performance evaluation in scientific big-data environments. They mounted file systems using FUSE clients and identified the characteristics of each file system through equivalent evaluations.…”

Section: Background and Related Workmentioning

confidence: 99%

Performance Analysis of Distributed File System Based on RAID Storage for Tapeless Storage

Kim,

Yu,

Kang

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

Tape storage is used as a long-term data storage solution in institutions and research centers around the world, contributing to the advancement of business and scientific research. Despite the existence of other storage media, tape storage has gained popularity owing to its low cost per storage capacity, making it an affordable option for initial implementation. However, recently, the strengths of tape storage have become ambiguous, and their drawbacks have become more evident. In the current storage system, tape storage requires complex management compared to other storage media, which incurs additional human resource costs, making it less cost effective. On the other side, other storage media continue to become cheaper and have larger storage capacities. Therefore, companies and research centers are no longer required to continuously purchase tape storage. With this trend, distributed file systems can be an alternative to tape storage. Distributed file systems combine storage nodes into a network through Reliable Array of Independent Nodes (RAIN). Erasure coding is used to store data in this file system, which makes the system cost-effective and highly reliable. This allows the construction of a safe and efficient storage system. In this study, we constructed and evaluated a distributed file system based on Redundant Array of Independent Disks (RAID) storage. We configured and evaluated distributed file systems that use Reed-Solomon in the same environment, providing the performance characteristics of each distributed file system.

show abstract

“…It is designed for compute-centric clusters without specific data-locality requirements. To leverage the use of commodity hardware, GlusterFS [23] provides different replication strategies, from simple replications to erasure coding, and eliminates the need for centralized metadata servers. Exploiting data locality with GlusterFS poses a challenge due to its design, which was not initially conceived for data-centric purposes.…”

Section: Related Workmentioning

confidence: 99%

“…FUSE or "file system in user space" is a library that allows the development of a file system interface without the need of dealing with delicate kernel adaptations. As with many other distributed file systems [16,23,24], AwareFS uses FUSE to implement functions for reading and writing data and for metadata manipulation. This easiness provides a regular Unix mountable file system interface (e.g., running commands like cat, cd, ls, rm, mkdir) and interaction with POSIX-compliant applications including proven benchmark applications such as IOR [33] and fio [34].…”

Section: Fuse Clientmentioning

confidence: 99%

Distributed File System to Leverage Data Locality for Large-File Processing

da Silva,

Sato,

Midorikawa

2023

Electronics

View full text Add to dashboard Cite

Over the past decade, significant technological advancements have led to a substantial increase in data proliferation. Both scientific computation and Big Data workloads play a central role, manipulating massive data and challenging conventional high-performance computing architectures. Efficiently processing voluminous files using cost-effective hardware remains a persistent challenge, limiting access to new technologies for individuals and organizations capable of higher investments. In response to this challenge, AwareFS, a novel distributed file system, addresses the efficient reading and updating of large files by consistently exploiting data locality on every copy. Its distributed metadata and lock management facilitate sequential and random I/O patterns with minimal data movement over the network. The evaluation of the AwareFS local-write protocol demonstrated efficiency across various update patterns, resulting in a performance improvement of approximately 13%, while benchmark assessments conducted across diverse cluster sizes and configurations underscored the flexibility and scalability of AwareFS. The innovative distributed mechanisms outlined herein are positioned to contribute to the evolution of emerging technologies related to the computation of data stored in large files.

show abstract

Performance Evaluations of Distributed File Systems for Scientific Big Data in FUSE Environment

Cited by 6 publications

References 14 publications

Visual Art Design of Digital Works Guided by Big Data

Visual Art Design of Digital Works Guided by Big Data

Performance Analysis of Distributed File System Based on RAID Storage for Tapeless Storage

Distributed File System to Leverage Data Locality for Large-File Processing

Contact Info

Product

Resources

About