IoT data cleaning techniques: A survey

Ding, Xiaoou; Wang, Hongzhi; Li, Genglong; Li, Haoxuan; Li, Yingze; Liu, Yida

doi:10.23919/icn.2022.0026

Cited by 11 publications

(4 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In IIoT scenarios, various sensing devices continuously produce a large amount of time-series data, and extracting value from such data requires the introduction of artificial intelligence, machine learning, etc., to support intelligent optimization and decision-making in IIoT systems. Different types of devices provide different data, first requiring different schemes for data cleaning to ensure data integrity and standardization [37]. Further, a wide variety of analysis algorithms are needed for different IoT application scenarios, such as face recognition, defect detection visual models [38,39], log processing natural language models [40], or anomaly detection, predictive analysis structured data processing models [41,42], each with unique development processes, integration methods, and usage.…”

Section: Intelligencementioning

confidence: 99%

IoT OS Platform: Software Infrastructure for Next-Gen Industrial IoT

Xing,

Lan,

Sun

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

With the rapid development of the Internet of Things (IoT), the growth of the Industrial Internet of Things (IIoT) applied in the industrial sector has also been swift. However, in practical applications, there are still issues such as the misalignment between theory and application, the lack of a unified standardization framework, and the frequent occurrence of data silos. These issues limit the maintainability and scalability of IIoT systems and increase the digitalization costs for enterprises. Based on this, drawing from the design principles of classic general-purpose operating systems, we propose the concept of an IoT operating system platform. As a software infrastructure aimed at the next generation of IIoT, the IoT OS platform consists of general-purpose computer operating systems and the platform software running on them. It manages computing resources and entities such as sensors, networks, and ubiquitous artificial intelligence applications or systems, and provides service support for IIoT applications upwards, aiming to improve existing issues and enhance the specificity and scalability of IIoT systems. This paper presents the current status of IIoT systems, the definition and architecture of the IoT OS platform, and validates the theoretical architecture through specific cases.

show abstract

Section: Intelligencementioning

confidence: 99%

IoT OS Platform: Software Infrastructure for Next-Gen Industrial IoT

Xing,

Lan,

Sun

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…It is also defined that the detection of anomalies is limited to the identification of anomalies, while data cleaning goes further to suppress the elements discovered. It has become a widely adopted technique for enterprise data management in data warehouses [104]. Data cleaning is a widespread topic of research in big data analysis [105].…”

Section: E Data Cleaningmentioning

confidence: 99%

Quality of Data (QoD) in Internet of Things (IOT): An Overview, State-of-the-Art, Taxonomy and Future Directions

Yalli,

Hasan,

Haron

et al. 2023

IJACSA

View full text Add to dashboard Cite

The Internet of Things (IoT) data is the main component for finding the basis that allows decisions to be made intelligently and enables other services to be explored and used. Data originates from smart things that have the capabilities to connect and share data enormously with other things in the IoT ecosystem. However, the level of intelligence obtained and the type of services provided, all depend on whether the data is trusted or not. High-quality data is the most trusted;, it can be used to extract meaningful insights from an event and can also be used to provide good services to humans. Therefore, decisions based on high-quality and trusted data could be good, whereas those based on low-quality or untrusted data are not only bad but could also have severe consequences. The term Quality of Data (QoD) is used to represent data trustworthiness and is used throughout this paper. To the best of our knowledge, this work is the first to coin the term QoD. The problems that hinder QoD are identified and discussed. One if it is an outlier, it is a major feature of the data that degrades its overall quality. Several machine-learning techniques that detect outliers have been studied and presented, with few data-cleaning techniques. This paper aims to present the elements necessary to ensure QoD by presenting the overview of the IoT state-of-the-art. Then, data quality, data in IoT, and outliers are studied, and some quality assurance techniques that maintain data quality is presented. A comprehensive taxonomy is shown to provide state-of-the-art data in IoT. Open issues and future directions were suggested at the end of the paper.

show abstract

“…Assessing the degradation behaviour solely based on a single factor fails to provide a comprehensive and accurate reflection. Secondly, these studies rely on complete and continuous power IoT sensing data, without considering the challenges posed by irregular data loss and reduced evaluation accuracy resulting from unstable communication channels and sensor failures during actual operation [16].…”

Section: Introductionmentioning

confidence: 99%

Evaluation method for insulation degradation of power transformer windings based on incomplete internet of things sensing data

Qu,

Zhao,

Zhao

et al. 2024

IET Science Measure & Tech

View full text Add to dashboard Cite

This paper proposes a novel evaluation method to address the challenge of evaluating insulation degradation in power transformer windings based on incomplete online Internet of Things (IoT) sensing data. The method leverages the Wasserstein Slim Generative Adversarial Imputation Network with Gradient Penalty algorithm to fill the irregularly missing power transformer IoT perception data, including voltage, current, temperature, and partial discharge. Subsequently, electrical, thermal, and mechanical performance degradation damage indicators for transformer winding insulation are constructed using the filled and complete IoT perception data. By applying the tensor fusion algorithm, the characteristics of these degradation damage indicators are fused, leading to the development of a comprehensive degradation evaluation index for the winding insulation. The evaluation of the winding insulation degradation state is achieved through the minimum quantization error method. The proposed method is validated using the real‐world transformer IoT perception data, and the experimental results demonstrate its ability to accurately assess the degree of winding insulation degradation, regardless of the presence of random or continuous irregularities in IoT sensing data.

show abstract

IoT data cleaning techniques: A survey

Cited by 11 publications

References 65 publications

IoT OS Platform: Software Infrastructure for Next-Gen Industrial IoT

IoT OS Platform: Software Infrastructure for Next-Gen Industrial IoT

Quality of Data (QoD) in Internet of Things (IOT): An Overview, State-of-the-Art, Taxonomy and Future Directions

Evaluation method for insulation degradation of power transformer windings based on incomplete internet of things sensing data

Contact Info

Product

Resources

About