A multi-tier data reduction mechanism for IoT sensors

Feng, Liang; Kortoçi, Pranvera; Liu, Yong

doi:10.1145/3131542.3131557

Cited by 17 publications

(13 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If T1=[]()t11,v11,()t21,v21,…,()tn1,vn1 is the real data and T2=[]()t12,v12,()t22,v22,…,()tn2,vn2 is the filtered data, the Jaccard similarity coefficient between them is calculated [4] as:α=true∑i=1nmin(ti1,vi2)true∑i=1nmax(ti1,vi2)×100 where n is the length of actual data. A range of maximum absolute error (etrueprefixmax = 0.01, 0.02,..., 0.09) was used to investigate the impact of e...…”

Section: Study Results and Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

An In-Networking Double-Layered Data Reduction for Internet of Things (IoT)

Ismael

Gao

Al-Shargabi

et al. 2019

Sensors

View full text Add to dashboard Cite

Due to the ever-increasing number and diversity of data sources, and the continuous flow of data that are inevitably redundant and unused to the cloud, the Internet of Things (IoT) brings several problems including network bandwidth, the consumption of network energy, cloud storage, especially for paid volume, and I/O throughput as well as handling huge amount of stored data in the cloud. These call for data pre-processing at the network edge before data transmission over the network takes place. Data reduction is a method for mitigating such problems. Most state-of-the-art data reduction approaches employ a single tier, such as gateways, or two tiers, such gateways and the cloud data center or sensor nodes and base station. In this paper, an approach for IoT data reduction is proposed using in-networking data filtering and fusion. The proposed approach consists of two layers that can be adapted at either a single tier or two tiers. The first layer of the proposed approach is the data filtering layer that is based on two techniques, namely data change detection and the deviation of real observations from their estimated values. The second layer is the data fusion layer. It is based on a minimum square error criterion and fuses the data of the same time domain for specific sensors deployed in a specific area. The proposed approach was implemented using Python and the evaluation of the approach was conducted based on a real-world dataset. The obtained results demonstrate that the proposed approach is efficient in terms of data reduction in comparison with Least Mean Squares filter and Papageorgiou’s (CLONE) method.

show abstract

Section: Study Results and Evaluationmentioning

confidence: 99%

“…These devices facilitate for applications the processing of a massive volume of heterogeneous data, including scalar data (e.g., temperature) and multi-media data (e.g., images), with high speed. This has given rise to many considerable network and cloud challenges [ 4 ].…”

Section: Introductionmentioning

confidence: 99%

An In-Networking Double-Layered Data Reduction for Internet of Things (IoT)

Ismael

Gao

Al-Shargabi

et al. 2019

Sensors

View full text Add to dashboard Cite

show abstract

“…Data Processing: Data processing can be done locally (where data is generated), using edge nodes, fog nodes, cloudlets, and cloud platforms [128]. Data filtering and aggregation are crucial operations in data processing [129]. Apache Storm, Spark, Rapid Miner, and High-Performance Computing Cluster (HPCC) are the tools used to process the data in real-time scenarios.…”

Section: ) Data Managementmentioning

confidence: 99%

“…c. Data Analysis: Data Analysis is the critical aspect to bring out the hidden patterns from the processed data [129]. This data is beneficial to control and actuate the decisions.…”

Section: ) Data Managementmentioning

confidence: 99%

An Empirical Study on System Level Aspects of Internet of Things (IoT)

2020

View full text Add to dashboard Cite

Internet of Things (IoT) is an integration of the Sensor, Embedded, Computing, and Communication technologies. The purpose of the IoT is to provide seamless services to anything, anytime at any place. IoT technologies play a crucial role everywhere, which brings the fourth revolution of disruptive technologies after the internet and Information and Communication Technology (ICT). The Research & Development community has predicted that the impact of IoT will be more than the internet and ICT on society, which improves the wellbeing of society and industries. Addressing the predominant system-level design aspects like energy efficiency, robustness, scalability, interoperability, and security issues result in the use of a potential IoT system. This paper presents the current state of art of the functional pillars of IoT and its emerging applications to motivate academicians and researches to develop real-time, energy-efficient, scalable, reliable, and secure IoT applications. This paper summarizes the architecture of IoT, with the contemporary status of IoT architectures. Highlights of the IoT system-level issues to develop more advanced real-time IoT applications have been discussed. Millions of devices exchange information using different communication standards, and interoperability between them is a significant issue. This paper provides the current status of the communication standards and application layer protocols used in IoT with the detailed analysis. The computing paradigms like Cloud, Cloudlet, Fog, and Edge computing facilitate IoT with various services like data offloading, resource and device management, etc. In this paper, an exhaustive analysis of Edge Computing in IoT with different edge computing architectures and existing status are deliberated. The widespread adoption of IoT in society has resulted in privacy and security issues. This paper emphasizes on analyzing the security challenges, privacy and security threats, conventional mitigation techniques, and further scope for IoT security. The features like fewer memory footprints, scheduling, real-time task execution, fewer interrupt, and thread switching latency of Real-Time Operating Systems (RTOS) enables the development of time critical IoT applications. Also, this review offers the analysis of the RTOS's suitable for IoT with the current status and networking stack. Finally, open research issues in IoT system development are discussed.

show abstract

“…These methods are used to decrease the overall data transmitted over the networks; this leads to a decrease in the sensors' energy consumption [7,8]. Compressing the data before the transmission process also leads to saving the bandwidth of the network; thus, the available resources can be used efficiently [9].…”

Section: Introductionmentioning

confidence: 99%

Internet of things data compression based on successive data grouping

Sawalha¹,

Al‐Naymat²

2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

Internet of things (IoT) is a useful technology in different aspects, and it is widely used in many applications; however, this technology faces some major challenges which need to be solved, such as data management and energy saving. Sensors generate a huge amount of data that need to be transferred to other IoT layers in an efficient way to save the energy of the sensor because most of the energy is consumed in the data transmission process. Sensors usually use batteries to operate; thus, saving energy is very important because of the difficulty of replacing batteries of widely distributed sensors. Reducing the total amount of transmitted data from the perception layer to the network layer in the IoT architecture will save the energy of the sensor. This paper proposes a new IoT data compression method; it is based on grouping similar successive data together and sending them as one row with a total number of occurrences. The decision of similarity is done by comparing the root-mean-square successive difference calculated on the training dataset. The evaluation of the proposed method was performed on the Intel Lab dataset and the compression performance of the proposed method was compared with other compression methods, where a great enhancement was achieved; the compression ratio was 10.953 with a reconstruction of temperature data error equal to 0.0313°C.

show abstract

A multi-tier data reduction mechanism for IoT sensors

Cited by 17 publications

References 95 publications

An In-Networking Double-Layered Data Reduction for Internet of Things (IoT)

An In-Networking Double-Layered Data Reduction for Internet of Things (IoT)

An Empirical Study on System Level Aspects of Internet of Things (IoT)

Internet of things data compression based on successive data grouping

Contact Info

Product

Resources

About