With the rapid development of science, technology, and engineering, large amounts of data have been generated in many fields in the past 20 years. In the process of medical research, data are constantly generated, and large amounts of real-world data form a “data disaster.” Effective data analysis and mining are based on data availability and high data quality. The premise of high data quality is the need to clean the data. Data cleaning is the process of detecting and correcting “dirty data,” which is the basis of data analysis and management. Moreover, data cleaning is a common technology for improving data quality. However, the current literature on real-world research provides little guidance on how to efficiently and ethically set up and perform data cleaning. To address this issue, we proposed a data cleaning framework for real-world research, focusing on the 3 most common types of dirty data (duplicate, missing, and outlier data), and a normal workflow for data cleaning to serve as a reference for the application of such technologies in future studies. We also provided relevant suggestions for common problems in data cleaning.

UNSTRUCTURED Real-world research inevitably leads to the generation of "dirty data", which can seriously impact data utilization and the quality of decision-making. Data cleaning is a critical method for improving data quality. However, the current literature surrounding real-world research provides little guidance on how to set up and carry out data cleaning efforts both efficiently and ethically. To address this issue, we propose a data cleaning framework for real-world research, focusing on the three most common types of "dirty data,” (duplicate data, missing data, and outlier data), as well as a normal workflow for data cleaning to provide a reference for the application of such technologies in future studies.