Distributed System Based on Correlation Sequencing Algorithm

Abstract

Due to the huge amount of data processed in the distributed storage system, and the transformation of tasks processed in the traditional stand-alone environment into a distributed processing mode, which increases the cost of network and scheduling, operations in traditional stand-alone databases are performed in distributed storage systems appear extra expensive in the system. With the development of high-throughput sequencing technologies, data analysis has increasingly introduced distributed technologies to address the challenges posed by the explosion of data. The main purpose of this paper is to design and study distributed systems based on correlation sequencing algorithms. In order to alleviate the relatively lagging development of this field in China, this paper builds a distributed computing and analysis platform for omics data analysis, and proposes an optimization method based on the retrieval and computing problems in omics data analysis. Experiments show that the MD5 value of the result file obtained after the error is recovered and the result file is compared with the result file without error is the same. It shows that the result obtained by the fault-tolerant method in this paper is correct. Finally, the effectiveness of the optimization is verified through specific test data.