Research on Public Management Application Innovation Based on Spark Big Data Framework

Abstract

Public management service is the key to urban intelligent construction. This paper proposes an analysis method and model based on Spark big data framework and takes resident income, happiness index, urban planning, and ecological environment as the indicators of Spark big data. From the high difficulty of Spark big data cluster analysis of urban public management, we build the index weight by the entropy weight method, optimize the similarity calculation, and achieve the rapid understanding of urban public management. Subsequently, the Spark big data public management platform is applied to the public management of Beijing. The results indicate that the public management platform based on Spark big data framework can improve the public management level of the city and help to build an intelligent city.

Scheduling of Jobs Allocation for Apache Spark Using Kubernetes for Efficient Execution of Big Data Application

“…Cloud computing is changing IT engineering by facilitating the flexible allocation or sharing of computing resources comprising of CPU, memory, storage, and networks, to create, customize, and optimize large-scale network and cloud database systems [4,14]. Since cloud computing serves many users around the world, large data centers host applications from different clients around the world [16]. These platforms use virtualization technology to duplicate the underlying physical resources due to which workloads for different applications vary greatly, workload and resource allocation must be controlled to ensure the quality of service [17].…”

“…The framework of Spark is designed explicitly to collaborate and run big data analysis for real-time applications [16,18,21]. It became a solid establishment for data learning and took the field of big data analytics at high speed.…”