Sakdas: A Python Package for Data Profiling and Data Quality Auditing

Loetpipatwanich, Sakda; Vichitthamaros, Preecha

doi:10.1109/ibdap50342.2020.9245455

Cited by 3 publications

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, several indicators state that a data has good quality, namely consistency, integrity, timeliness, and usability [4], [5], [6]. Without good quality, companies cannot make good decisions because of ignorance of the conditions that occur in the company [7].…”

Section: Introductionmentioning

confidence: 99%

Comparison of Apache SparkSQL and Oracle Performance: Case Study of Data Cleansing Process

Hidayati

Kusumasari²,

Hamami³

2022

JOIV : Int. J. Inform. Visualization

View full text Add to dashboard Cite

A dataset with good quality is a valuable asset for a company. The data can be processed into information to help companies improve decision-making. However, the data increased more and more over time to decrease data quality. Thus, good data management is important to keep data quality meeting company standards. One of the efforts that can be done is conducting data cleansing to clean data from errors, inaccuracies, duplication, format discrepancies, etc. Apache Spark is an engine that can analyze large amounts of data. Oracle Database is a database management system used to manage databases. Both have their own reliability and can be used to analyze SQL-shaped data. This study compared Spark and Oracle performance based on query processing time. Both were tested on queries used to perform data cleansing of millions of rows of the dataset. The research focuses on finding out Spark and Oracle's performance through quantitative analysis. The results of this study showed that there were differences in query processing times on both tools. Apache Spark is rated better because it has a relatively faster query processing time than Oracle Database. It can be concluded that Oracle is more reliable in storing complex data models than in analyzing large data. For future research, it is suggested to add other comparison aspects such as memory and CPU usage. The researchers can also consider using query optimization techniques to enrich query experiments.

show abstract

Section: Introductionmentioning

confidence: 99%

Comparison of Apache SparkSQL and Oracle Performance: Case Study of Data Cleansing Process

Hidayati

Kusumasari²,

Hamami³

2022

JOIV : Int. J. Inform. Visualization

View full text Add to dashboard Cite

show abstract

Data Driven Smart Cities and Data Spaces

Battistoni

2023

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

Smart Cities are places where traditional networks and services are made more efficient through digital solutions, benefiting residents and businesses. Moving to digital solutions basically means applying a technological process that transforms various aspects of social or individual life into data that are subsequently transformed into information endowed with new forms of value, including economic value. In addition to economic value, in the digital age, access to data makes it possible to have transparent and open government, build trust, fight corruption, address inequality, and create more resilient democracies. This chapter describes how, for these purposes, data management and sharing are changing with the adoption of Data Spaces, a new approach that moves away from data integration to data coexistence.

show abstract

Service for Monitoring the Conditions of Vaccination of the Population from Coronavirus Infection

Feshchenko¹,

Yurchak²

2022

ACPS

View full text Add to dashboard Cite

Health care is a system of measures aimed at ensuring the preservation and development of physiological and psychological functions, optimal performance and social activity of man at the maximum biologically possible individual life expectancy. The key goal of any health care system is the availability and quality of medical care, medical services, medical care. The main goal of any country is to build an effective model of the health care system and achieve the highest level of public health with minimal resources. The purpose of developing a service to monitor the vaccination status of the population against coronavirus infection is to help users learn information about vaccination and general statistics in the country on vaccinations. This system is suitable for both qualified medical users and users who are unfamiliar with medicine but want to know more about vaccination data. The article attempts to analyze the types of services for vaccinations and their alternatives. It describes the principle of operation of the vaccination service in the form of a report made on the application platform for visualization of Microsoft Power BI Desktop.

show abstract

Sakdas: A Python Package for Data Profiling and Data Quality Auditing

Cited by 3 publications

References 11 publications

Comparison of Apache SparkSQL and Oracle Performance: Case Study of Data Cleansing Process

Comparison of Apache SparkSQL and Oracle Performance: Case Study of Data Cleansing Process

Data Driven Smart Cities and Data Spaces

Service for Monitoring the Conditions of Vaccination of the Population from Coronavirus Infection

Contact Info

Product

Resources

About