A hybrid disaster-tolerant model with DDF technology for MooseFS open-source distributed file system

Li, Xigao; Lin, Qian

doi:10.1007/s11227-016-1902-9

Cited by 5 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most of the lightweight DFSs strongly rely on the RAID or external software supports [4,9]. Due to the limit in either performance or disaster recover range, these methods are not applicable in the larger scale of smart grid power data, and we are unable to make the comparison with our proposed model.…”

Section: Disaster-tolerance In Lightweight Dfsmentioning

confidence: 99%

“…Our former work of a disaster-tolerance model for a specially distributed file system, MooseFS, using the Direct Data Fetch (DDF) technology, which solved the transfer suspension problem during the disaster happens [4,8]. The DDF technology will seek partially file index from data node, and directly send the data to remote data center.…”

Section: B Direct Data Fetch Technologymentioning

confidence: 99%

See 1 more Smart Citation

Get Rid of Data Disaster: An Active-active Model with VRN Network for Smart Grid DFS

Yu¹,

Li²,

Pang³

et al. 2017

Proceedings of the International Conference on Computer Networks and Communication Technology (CNCT 2016)

Self Cite

View full text Add to dashboard Cite

Abstract. Data storage security issues are expanding to global area, while -Internet+‖ Smart Energy is prospering in high speed. Therefore, the security backup for the distributed file system (DFS) has become the key factor in the construction of smart grid in electric power. However, due to remote data synchronization across multiple data centers, the reliability and robustness of the storage system is relatively weak. Our work proposed a novel active-active disaster recovery model, with a Virtual Redirection Network (VRN). Under the automatic control of VRN, the model provides a solution of data synchronization across large scope of power grid center. The results show that the system not only improves the IO performance in massive small files, but also increases the total system robustness over a long distance data transfer.

show abstract

Section: Disaster-tolerance In Lightweight Dfsmentioning

confidence: 99%

Section: B Direct Data Fetch Technologymentioning

confidence: 99%

Get Rid of Data Disaster: An Active-active Model with VRN Network for Smart Grid DFS

Yu¹,

Li²,

Pang³

et al. 2017

Proceedings of the International Conference on Computer Networks and Communication Technology (CNCT 2016)

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is one of the most promising potential solutions for storing massive amounts of remote sensing image data. Currently, the mainstream distributed file systems mainly include HDFS [10], Lustre [11], FastDFS [12], GridFS [13], MooseFS [14], GlusterFS [15] and CEPH [16]. FastDFS, GridFS and GlusterFS are suitable for file-based online services, such as video and still images.…”

Section: Introductionmentioning

confidence: 99%

Efficient Management and Scheduling of Massive Remote Sensing Image Datasets

Zhu

Zhang

Zhao

et al. 2023

IJGI

View full text Add to dashboard Cite

The rapid development of remote sensing image sensor technology has led to exponential increases in available image data. The real-time scheduling of gigabyte-level images and the storage and management of massive image datasets are incredibly challenging for current hardware, networking and storage systems. This paper’s three novel strategies (ring caching, multi-threading and tile-prefetching mechanisms) are designed to comprehensively optimize the remote sensing image scheduling process from image retrieval, transmission and visualization perspectives. A novel remote sensing image management and scheduling system (RSIMSS) is designed using these three strategies as its core algorithm, the PostgreSQL database and HDFS distributed file system as its underlying storage system, and the multilayer Hilbert spatial index and image tile pyramid to organize massive remote sensing image datasets. Test results show that the RSIMSS provides efficient and stable image storage performance and allows real-time image scheduling and view roaming.

show abstract

FastCache: A Client-Side Cache with Variable-Position Merging Schema in Network Storage System

Lin

Wang

et al. 2022

Algorithms and Architectures for Parallel Processing

View full text Add to dashboard Cite

A hybrid disaster-tolerant model with DDF technology for MooseFS open-source distributed file system

Cited by 5 publications

References 6 publications

Get Rid of Data Disaster: An Active-active Model with VRN Network for Smart Grid DFS

Get Rid of Data Disaster: An Active-active Model with VRN Network for Smart Grid DFS

Efficient Management and Scheduling of Massive Remote Sensing Image Datasets

FastCache: A Client-Side Cache with Variable-Position Merging Schema in Network Storage System

Contact Info

Product

Resources

About