Optimized Data Replication for Small Files in Cloud Storage Systems

Xiong, Fei; Liu, Wenjie; Yeliang, Cang; Gong, Xiaofeng; Deng, Song

doi:10.1155/2016/4837894

Cited by 16 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research is extending based on cognitive psychology theory, showing promising results when used with appropriate manipulators. Suggestions have been made for optimising human-machine collaboration, and defining ethics in working with machines can lead to a successful relationship between humans and machines, resulting in increased work efficiency [35]. However, limited studies also highlight unethical behaviour towards machines, emphasising the need for setting boundaries and defining ethics in working with machines [36].…”

Section: Dap-framework Representationmentioning

confidence: 99%

A Conceptual Evaluation of the Data as Partner (DAP) Framework

Hassan,

Sobnath,

Snell

et al. 2023

2023 IEEE International Smart Cities Conference (ISC2)

View full text Add to dashboard Cite

This paper explores the innovative Data As Partner (DAP) philosophy as a transformative approach to organisational data strategy, drawing upon established concepts of information value, technology acceptance, and inclusivity for people with disabilities (PwD) in smart cities. The study conducted a comprehensive systematic review, retrieving 380 relevant documents from diverse databases using specific keywords for precision. After meticulous screening, 20 eligible papers were selected, forming the foundation for deriving the key components of the DAP framework. Subsequently, a qualitative focus group study engaged 18 participants from diverse sectors to evaluate the practical applicability of the DAP conceptual framework. Thematic analysis of the focus group sessions revealed five key themes: practical value, pragmatic performance, ethical perspective, clarity of the framework, and enhancement tools. These findings offer valuable insights for refining the DAP framework and optimising its integration into real-world scenarios, particularly in empowering individuals in smart cities, especially those with disabilities. Future research should explore larger and more diverse samples to generalise findings and assess the framework's long-term impact through longitudinal evaluations. By adopting user-centred methodologies, the DAP framework can be further enhanced as a strategic tool, empowering data to actively participate in organisational decision-making processes.

show abstract

Section: Dap-framework Representationmentioning

confidence: 99%

A Conceptual Evaluation of the Data as Partner (DAP) Framework

Hassan,

Sobnath,

Snell

et al. 2023

2023 IEEE International Smart Cities Conference (ISC2)

View full text Add to dashboard Cite

show abstract

“…Yanfeng Lyu et al [15] proposed an efficient merging method that substantially reduces the access time for small files by using caching and prefetching methods. X. Fu et al [16] proposed a block replica placement technique for effectively processing small files where files are merged as per the predetermined parameters. Qi Mu et al [17] advised a method for dealing with small files that is both efficient and effective.…”

Section: Literature Reviewmentioning

confidence: 99%

Optimization of Small Sized File Access Efficiency in Hadoop Distributed File System by Integrating Virtual File System Layer

Alange¹,

Mathur²

2022

IJACSA

View full text Add to dashboard Cite

Storage for large datasets, handling data in different formats and data getting generated with high speed are the major highlights of the Hadoop because of which the Hadoop got invented. Hadoop is the solution for the big data problems as discussed above. In order to give the improved solution (in terms of access efficiency and time) for small sized files, this solution is proposed. A novel approach called VFS-HDFS architecture is designed in which the focus is on optimization of small sized files access problems with significant development compared with the existing solutions i.e. HDFS sequence files, HAR, NHAR. In the proposed work a Virtual file system layer has been added as a wrapper over the top of existing HDFS architecture. However, the research work is carried out without altering the existing HFDS architecture. In this paper drawbacks of existing techniques i.e. Flat File Technique and Table Chain Technique which are implemented in HDFS HAR, NHAR, sequence file is overcome by using Bucket Chain Technique. The files to merge in a single bucket are selected using ensemble classifier which is a combination of different classifiers. Combination of multiple classifiers gives the better accurate results. Using this proposed system, better results are obtained compared with the existing system in terms of access efficiency of small sized files in HDFS.

show abstract

“…According to the description, this method can effectively reduce free space fragmentation even after long‐term ageing. The work of Fu et al proposes an optimized data replication approach for small files in cloud storage systems. Small files are classified into four types according to their access frequencies.…”

Section: Related Workmentioning

confidence: 99%

Weight‐based strategy for an I/O‐intensive application at a cloud data center

Peng

Liu

Wang

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Applications with different characteristics in the cloud may have different resources preferences.However, traditional resource allocation and scheduling strategies rarely take into account the characteristics of applications. Considering that an I/O-intensive application is a typical type of application and that frequent I/O accesses, especially small files randomly accessing the disk, may lead to an inefficient use of resources and reduce the quality of service (QoS) of applications, a weight allocation strategy is proposed based on the available resources that a physical server can provide as well as the characteristics of the applications. Using the weight obtained, a resource allocation and scheduling strategy is presented based on the specific application characteristics in the data center. Extensive experiments show that the strategy is correct and can guarantee a high concurrency of I/O per second (IOPS) in a cloud data center with high QoS. Additionally, the strategy can efficiently improve the utilization of the disk and resources of the data center without affecting the service quality of applications. KEYWORDS IOPS, process scheduling, random I/O, small files, weight INTRODUCTIONIn recent years, cloud computing has been widely used because of its advantages, such as its high scalability, low energy-consumption, and on-demand provision of virtual resources. Currently, companies such as Google, IBM, Amazon, Microsoft, and Alibaba are the leaders in the development and application of cloud computing technology and services. 1 As a new business mode, cloud computing provides computing, storage, and network services to Internet users by integrating resources. As a result, users conveniently make use of their applications with much lower purchase and maintenance costs. Additionally, cloud computing can greatly improve the resource utilization efficiency of society. Therefore, many enterprises as well as personal users have moved their applications to the cloud environment. In cloud computing, the data center is the key feature, as it owns a large amount of the resource pool that provides powerful resources and serves all types of applications.However, although the resources in a cloud data center are enormous, they are limited. Different consumptions of resources might considerably affect the performance and service capability of a cloud data center. Traditionally, resources are generally allocated based on a virtual machine (VM).Applications are executed on the virtual machine without considering their characteristics or features. Actually, different types of applications have different characteristics and consume different types and amounts of resources during their execution. These resource requirement difference among applications leads to serious problems for cloud resource allocations and task schedule strategies. For example, some types of resources may have been highly overloaded while other types of resources are barely consumed in different virtual machines or by different applications. This occur...

show abstract

Optimized Data Replication for Small Files in Cloud Storage Systems

Cited by 16 publications

References 11 publications

A Conceptual Evaluation of the Data as Partner (DAP) Framework

A Conceptual Evaluation of the Data as Partner (DAP) Framework

Optimization of Small Sized File Access Efficiency in Hadoop Distributed File System by Integrating Virtual File System Layer

Weight‐based strategy for an I/O‐intensive application at a cloud data center

Contact Info

Product

Resources

About