A Reference Architecture and Model for Sensor Data Warehousing

Dobson, Simon; Golfarelli, Matteo; Graziani, Simone; Rizzi, Stefano

doi:10.1109/jsen.2018.2861327

Cited by 11 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Agricultural producers, consultants, and others who install, manage, or use such IoT monitoring systems for decision making can adopt these best practices in recording data from IoT systems that will allow for more efficient use of storage space and data organization. More information on database management can be found in Dobson et al 2018.…”

Section: Discussionmentioning

confidence: 99%

How to structure data from an IoT monitoring network

Zhang¹

2020

EDIS

View full text Add to dashboard Cite

Internet of Things (IoT) has been widely used in agriculture for smart farming and precision agriculture to improve efficiency and effectiveness in a data-oriented fashion. Monitored data is collected at a much higher frequency and in relatively higher reliability and accuracy than those of the manually collected information. However, there is a challenge with such large data sets in how the data is stored and processed to keep the information completeness and meet the dynamic requirements for future development and usage. The data from such systems typically consists readings from monitoring sensors and corresponding attributes, such as location, units, measurement types (e.g. air temperature, wind speed, air pressure) for each measurement. Problems such as huge data redundancy, loss of information, and inflexible data structure may occur and cost substantial time and investment for IoT system managers who do not have the necessary professional training and are not using any IT consulting service. This publication discusses the typical attributes in IoT monitoring networks and presents a data structure that outlines best practices in organizing data. The main goal is to provide knowledge on how such a system’s data should be structured so that these IoT monitoring network owners, such as agricultural producers, can optimize how they use their data. Thus, the primary audience is agricultural producers, consultants and others who install, manage, or use such IoT monitoring systems for decision making.

show abstract

Section: Discussionmentioning

confidence: 99%

How to structure data from an IoT monitoring network

Zhang¹

2020

EDIS

View full text Add to dashboard Cite

show abstract

“…The meta-data help to speed up the process by adding some additional data, which describe the stored data. For instance Dobson et al in [ 50 ] presented a reference model for the sensor and data warehouse. They employ advanced data warehousing methods to create an extendable model.…”

Section: Next-generation Advancements In the Internet Of Things (Iot)...mentioning

confidence: 99%

A Step toward Next-Generation Advancements in the Internet of Things Technologies

Amin

Abbasi

Mateen

et al. 2022

Sensors

View full text Add to dashboard Cite

The Internet of Things (IoT) devices generate a large amount of data over networks; therefore, the efficiency, complexity, interfaces, dynamics, robustness, and interaction need to be re-examined on a large scale. This phenomenon will lead to seamless network connectivity and the capability to provide support for the IoT. The traditional IoT is not enough to provide support. Therefore, we designed this study to provide a systematic analysis of next-generation advancements in the IoT. We propose a systematic catalog that covers the most recent advances in the traditional IoT. An overview of the IoT from the perspectives of big data, data science, and network science disciplines and also connecting technologies is given. We highlight the conceptual view of the IoT, key concepts, growth, and most recent trends. We discuss and highlight the importance and the integration of big data, data science, and network science along with key applications such as artificial intelligence, machine learning, blockchain, federated learning, etc. Finally, we discuss various challenges and issues of IoT such as architecture, integration, data provenance, and important applications such as cloud and edge computing, etc. This article will provide aid to the readers and other researchers in an understanding of the IoT’s next-generation developments and tell how they apply to the real world.

show abstract

“…3.2 Data Lake 3.2.1 A Reference Architecture and Model for Sensor Data Warehousing. In the paper (Dobson et al, 2018), the authors propose an architecture and data warehouse model for analyzing sensor data. The architecture is currently in the status of a proposal and has not been implemented in real use cases.…”

Section: Modelling and Implementing Big Data Warehouses For Decision mentioning

confidence: 99%

“…Data maintenance is supported by ETL processes in Big Data warehousing architectures. Data lakes (Dobson et al, 2018;Hai et al, 2016;Zhuang et al, 2016) accumulate source data in its' original format. Virtual integration architectures (ElSheikh et al, 2013;Yuan et al, 2010) do not store copies of source data, therefore, maintenance problem is solved naturally.…”

Section: Evolutionmentioning

confidence: 99%

“…Iterative model enrichment Iterative model enrichment (Dobson et al, 2018) Yes (web service description has to be updated dynamically) (Yuan et al, 2010) Naturally supported N/A N/A (Gadepally et al, 2017) N/A Additional islands or database engines can be supported N/A New data for base relations are added with union Additional database engines can be supported N/A (Alsubaiee et al, 2014) Inserts and deletes Schema needs to be defined Partially (Nadal et al, 2017(Nadal et al, , 2019 Provided by batch and speed layers A new release is added to the ontology A new release is added to the ontology (Vanhove et al, 2015) N/A New data sources are included automatically by Enterprise service bus using live transformation between two data stores N/A…”

Section: Evolutionmentioning

confidence: 99%

See 1 more Smart Citation

Handling Evolution in Big Data Architectures

Solodovnikova¹,

Niedrite²

2020

BJMC

View full text Add to dashboard Cite

One of the purposes of Big Data systems is to support analysis of data gathered from heterogeneous data sources. Since data warehouses have been used to achieve the same goal, they could be leveraged also to provide analysis of Big Data. The problem of adapting data warehouse data and schemata to changes in user requirements and data sources has been studied by many researchers worldwide. However, innovative methods must be developed also to support evolution in Big Data systems. In this paper, we analyze architectures designed for Big Data processing and analysis described in the literature with the purpose to identify the most appropriate solution for the evolution problem. We concentrate on four architecture types: data lakes, virtual integration, polystores, and λ-architecture, and, in addition to them, we consider solutions that apply data warehouse/OLAP methods to Big Data processing and analysis. Finally, we describe our proposal of an architecture that allows to perform different kinds of analytical tasks on Big Data retrieved from multiple heterogeneous data sources with different latency and is capable of processing changes in data sources as well as evolving analysis requirements.

show abstract

A Reference Architecture and Model for Sensor Data Warehousing

Cited by 11 publications

References 17 publications

How to structure data from an IoT monitoring network

How to structure data from an IoT monitoring network

A Step toward Next-Generation Advancements in the Internet of Things Technologies

Handling Evolution in Big Data Architectures

Contact Info

Product

Resources

About