Towards Real-Time Processing of Massive Spatio-temporally Distributed Sensor Data: A Sequential Strategy Based on Kriging

Lorkowski, Peter; Brinkhoff, Thomas

doi:10.1007/978-3-319-16787-9_9

Cited by 3 publications

(7 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although kriging delivers an unbiased estimate and a type of error map (kriging variance), its computational complexity makes it an unlikely spatial interpolation method for integrating up to 100k moving sensor streams in real time without using massive computational resources. Srinivasan et al (2010), Lorkowski and Brinkhoff (2015a), and Zhong et al (2016) have adapted kriging for data streams; this work has confirmed that kriging does not scale to large numbers of sensors, with kriging taking 2 s to process 20 tuples using a 50 × 50 raster grid (Zhong et al, 2016) and 30 s to process 10 tuples using a 150 × 150 raster grid (Lorkowski & Brinkhoff, 2015a, 2015b, respectively.…”

Section: Scalability Of Spatial Interpolation Methodsmentioning

confidence: 67%

“…Srinivasan et al. (), Lorkowski and Brinkhoff (2015a), and Zhong et al. () have adapted kriging for data streams; this work has confirmed that kriging does not scale to large numbers of sensors, with kriging taking 2 s to process 20 tuples using a 50 × 50 raster grid (Zhong et al., ) and 30 s to process 10 tuples using a 150 × 150 raster grid (Lorkowski & Brinkhoff, 2015a, 2015b), respectively.…”

Section: Introductionmentioning

confidence: 70%

“…The computational complexity of a spatial interpolation method has a significant impact on its scalability (i.e., its potential to write algorithms that still perform efficiently although the number of input points increases significantly). For instance, typical implementations of kriging (Krige, 1951) have a computational complexity of O(n 4 ) or O(n 3 ) if improved (Lorkowski & Brinkhoff, 2015b;Zhong et al, 2016). Although kriging delivers an unbiased estimate and a type of error map (kriging variance), its computational complexity makes it an unlikely spatial interpolation method for integrating up to 100k moving sensor streams in real time without using massive computational resources.…”

Section: Scalability Of Spatial Interpolation Methodsmentioning

confidence: 99%

“…However, kriging is computationally expensive, since it requires computing a variogram, fitting a model to the variogram, and the kriging process itself, which has a computational complexity of O ( n 3 ) or O ( n 4 ) with n observations. Efforts have been made to adapt kriging for data streams (Lorkowski & Brinkhoff, 2015a; Srinivasan, Duraiswami, & Murtugudde, ; Zhong, Kealy, & Duckham, ), but still the computational complexity of the kriging step itself severely limits high throughput. Thus, kriging is a less likely candidate for integrating up to 250k sensors in real time.…”

Section: Introductionmentioning

confidence: 99%

“…As mentioned previously, Srinivasan et al. (), Lorkowski and Brinkhoff (2015a), and Zhong et al. () have adapted kriging for data streams processing using a stream‐based operator paradigm.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Real‐time inverse distance weighting interpolation for streaming sensor data

Liang

Nittel

Whittier

et al. 2018

Transactions in GIS

View full text Add to dashboard Cite

With advances in technology and an increasing variety of inexpensive geosensors, environmental monitoring has become increasingly sensor dense and real time. Using sensor data streams enables real‐time applications such as environmental hazard detection, or earthquake, wildfire, or radiation monitoring. In‐depth analysis of such spatial fields is often based on a continuous representation. With very large numbers of concurrent observation streams, novel algorithms are necessary that integrate streams into rasters, or other continuous representations, continuously in real time. In this article, we present an approach leveraging data stream engines (DSEs) to achieve scalable, high‐throughput inverse distance weighting (IDW). In detail, we designed and implemented a novel stream query operator framework that extends general‐purpose DSEs. The proposed framework includes a two‐panel, spatio‐temporal grid‐based index and several algorithms, namely the Shell and k‐Shell algorithms, to estimate individual grid cells efficiently and adaptively for different sampling scenarios. For our performance experiments, we generated several different spatio‐temporal stream data sets based on the radiation deposits in the Fukushima region after the nuclear accident of 2011 in Japan. Our results showed that the k‐Shell algorithm of the proposed framework produces a raster based on 250k observation streams in under 0.5 s using a state‐of‐the‐art workstation.

show abstract

Section: Scalability Of Spatial Interpolation Methodsmentioning

confidence: 67%

Section: Introductionmentioning

confidence: 70%

Section: Scalability Of Spatial Interpolation Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Real‐time inverse distance weighting interpolation for streaming sensor data

Liang

Nittel

Whittier

et al. 2018

Transactions in GIS

View full text Add to dashboard Cite

show abstract

Spatially Representative Online Big Data Sampling for Smart Cities

Jawarneh

Bellavista

Corradi

et al. 2020

2020 IEEE 25th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD)

View full text Add to dashboard Cite

Environmental monitoring of continuous phenomena by sensor data streams: A system approach based on Kriging

Lorkowski¹,

Brinkhoff²

2015

Advances in Computer Science Research

Self Cite

View full text Add to dashboard Cite

Environmental monitoring as technological endeavour has to deal with limitations in respect to transmission capacities, computational resources and storage space. Despite the progress in ICT, those limitations remain significant because of rising demands like real-time monitoring on the one hand and increased amount of observations on the other. The geostatistic method of kriging, originally developed to manage spatial uncertainty, provides elaborate means for spatio-temporal interpolation and is therefore the method of choice wherever continuous phenomena are to be monitored by discrete observations. With its associated confidence estimation, the method can also be exploited to solve other problems like fusion of sub-models for continuous real-time updates or filtering massive sensor streams. In this work, we suggest a system, substantially based on kriging, to filter, process, interpolate, monitor and archive sensor data streams. We incorporate several algorithmic and technical solutions into this system and evaluate them by simulated and real data.

show abstract

Towards Real-Time Processing of Massive Spatio-temporally Distributed Sensor Data: A Sequential Strategy Based on Kriging

Cited by 3 publications

References 13 publications

Real‐time inverse distance weighting interpolation for streaming sensor data

Real‐time inverse distance weighting interpolation for streaming sensor data

Spatially Representative Online Big Data Sampling for Smart Cities

Environmental monitoring of continuous phenomena by sensor data streams: A system approach based on Kriging

Contact Info

Product

Resources

About