A Stream Processing System for Multisource Heterogeneous Sensor Data

Hu, Liang; Sun, Rui; Wang, Feng; Fei, Xiuhong; Zhao, Kuo

doi:10.1155/2016/4287834

Cited by 8 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to address the challenge of scalable processing and low latency for IoT cloud, a robotic application is developed by [72]. [73] in another study setup a real time system for processing heterogeneous sensor streams from multiple sources with low latency where Apache Storm is responsible for distributed real time sensor data processing. Authors in [74] proposed a framework to address challenge of continuous growth of massive data streams in a smart city network.…”

Section: B Assessment Of Rq2: Which Challenges Have Been Faced Durinmentioning

confidence: 99%

Challenges and Solutions for Processing Real-Time Big Data Stream: A Systematic Literature Review

Mehmood

Anees

2020

IEEE Access

View full text Add to dashboard Cite

Contribution: Recently, real-time data warehousing (DWH) and big data streaming have become ubiquitous due to the fact that a number of business organizations are gearing up to gain competitive advantage. The capability of organizing big data in efficient manner to reach a business decision empowers data warehousing in terms of real-time stream processing. A systematic literature review for real-time stream processing systems is presented in this paper which rigorously look at the recent developments and challenges of real-time stream processing systems and can serve as a guide for the implementation of real-time stream processing framework for all shapes of data streams. Background: Published surveys and reviews either cover papers focusing on stream analysis in applications other than real-time DWH or focusing on extraction, transformation, loading (ETL) challenges for traditional DWH. This systematic review attempts to answer four specific research questions. Research Questions: 1)Which are the relevant publication channels for real-time stream processing research? 2) Which challenges have been faced during implementation of real-time stream processing? 3) Which approaches/tools have been reported to address challenges introduced at ETL stage while processing real-time stream for real-time DWH? 4) What evidence have been reported while addressing different challenges for processing real-time stream? Methodology: A systematic literature was conducted to compile studies related to publication channels targeting real-time stream processing/joins challenges and developments. Following a formal protocol, semi-automatic and manual searches were performed for work from 2011 to 2020 excluding research in traditional data warehousing. Of 679,547 papers selected for data extraction, 74 were retained after quality assessment. Findings: This systematic literature highlights implementation challenges along with developed approaches for real-time DWH and big data stream processing systems and provides their comparisons. This study found that there exists various algorithms for implementing real-time join processing at ETL stage for structured data whereas less work for un-structured data is found in this subject matter. INDEX TERMS Real-time stream processing, big data streaming, structured/un-structured data, ETL, systematic literature review Publication source Channel References No. %age IEEE Transactions on Knowledge and Data Engineering

show abstract

Section: B Assessment Of Rq2: Which Challenges Have Been Faced Durinmentioning

confidence: 99%

Challenges and Solutions for Processing Real-Time Big Data Stream: A Systematic Literature Review

Mehmood

Anees

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…A flexible architecture can be achieved, as discussed inReference [53]. All applications can benefit from new, low-power WSN standards and platforms, as illustrated in References [47,48,49,50,51]. By taking them into account, a modular IoT architecture is proposed in Reference [4].…”

Section: Related Workmentioning

confidence: 99%

Design of Wireless Sensors for IoT with Energy Storage and Communication Channel Heterogeneity

Borza

Machedon-Pisu

Hamza-Lup

2019

Sensors

View full text Add to dashboard Cite

Autonomous Wireless Sensors (AWSs) are at the core of every Wireless Sensor Network (WSN). Current AWS technology allows the development of many IoT-based applications, ranging from military to bioengineering and from industry to education. The energy optimization of AWSs depends mainly on: Structural, functional, and application specifications. The holistic design methodology addresses all the factors mentioned above. In this sense, we propose an original solution based on a novel architecture that duplicates the transceivers and also the power source using a hybrid storage system. By identifying the consumption needs of the transceivers, an appropriate methodology for sizing and controlling the power flow for the power source is proposed. The paper emphasizes the fusion between information, communication, and energy consumption of the AWS in terms of spectrum information through a set of transceiver testing scenarios, identifying the main factors that influence the sensor node design and their inter-dependencies. Optimization of the system considers all these factors obtaining an energy efficient AWS, paving the way towards autonomous sensors by adding an energy harvesting element to them.

show abstract

“…Such edge-centric processing is important in real-time, mission-critical applications such as self-driving vehicles [7]. Significant examples of edge-centric processing are: [1], [8], [21], [23], [27], [14]. Some efforts deal with the automated separation of queries graphs into two sets [10]: queries processed at the edge devices and queries processed in the cloud.…”

Section: Related Workmentioning

confidence: 99%

Knowledge-centric Analytics Queries Allocation in Edge Computing Environments

Sagkriotis

Kolomvatsos

Anagnostopoulos

et al. 2019

2019 IEEE Symposium on Computers and Communications (ISCC)

View full text Add to dashboard Cite

The Internet of Things (IoT) involves a huge number of devices that collect data and deliver them to the Cloud. The processing of data at the Cloud is characterized by increased latency in providing responses to analytics queries defined by analysts or applications. Hence, Edge Computing (EC) comes into the scene to provide data processing close to the source. The collected data can be stored in edge devices and queries can be executed there to reduce latency. In this paper, we envision a case where entities located in the Cloud undertake the responsibility of receiving analytics queries and decide on the most appropriate edge nodes for queries execution. The decision is based on statistical signatures of the datasets of nodes and the statistical matching between statistics and analytics queries. Edge nodes regularly update their statistical signatures to support such decision process. Our performance evaluation shows the advantages and the shortcomings of our proposed schema in edge computing environments.

show abstract

A Stream Processing System for Multisource Heterogeneous Sensor Data

Cited by 8 publications

References 32 publications

Challenges and Solutions for Processing Real-Time Big Data Stream: A Systematic Literature Review

Challenges and Solutions for Processing Real-Time Big Data Stream: A Systematic Literature Review

Design of Wireless Sensors for IoT with Energy Storage and Communication Channel Heterogeneity

Knowledge-centric Analytics Queries Allocation in Edge Computing Environments

Contact Info

Product

Resources

About