Dual-carbon data fusion cleaning model based on Kafka cluster

Abstract

The conventional cleaning model of dual-carbon data fusion is more inclined to data cleaning, and there are mistakes in attribute judgment during data fusion, which leads to the problems of missing and duplication of the output cleaning data, which affects the application of dual-carbon data. Therefore, this paper studies the dual-carbon data fusion cleaning model based on Kafka cluster. Select the comprehensive attribute weights of the dual-carbon data fusion cleaning model, fuse the data with similar characteristics according to the characteristics of the data attributes, and find out the missing values in the process of repeated cleaning, so as to avoid the data fusion deviation. Based on Kafka cluster, the regularization term of dual-carbon data fusion cleaning target is constructed to reduce the dimension of dual-carbon data, and a regularization term is constructed on the basis of data fusion cleaning target function to make the fused dualcarbon data more meaningful. The comparison experiment shows that the cleaning performance of the model is better and it can be applied to real life.