The Open Source DataTurbine Initiative: Empowering the Scientific Community with Streaming Data Middleware

Fountain, Tony; Tilak, Sameer; Shin, Peter; Nekrasov, Michael

doi:10.1890/0012-9623-93.3.242

Cited by 10 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Data Collection: Incorporating Information Management Earlymentioning

confidence: 99%

“…The toolbox generates a log file of all operations performed for inclusion in the metadata to document any data transformations. EDGE also employs the Open Source DataTurbine (OSDT) server -a real-time streaming data engine that receives data from sensors and then transmits that data to other programs, such as Real-Time Data Viewer (RDV) -to examine sensor performance (Fountain et al 2012). The RDV provides an interface for viewing time-synchronized plots of the data, thus facilitating the detection of anomalous patterns that indicate sensor malfunctions (Figure 2; Daugherty et al 2011).…”

Section: Macrosystems Ecologymentioning

confidence: 99%

See 1 more Smart Citation

Completing the data life cycle: using information management in macrosystems ecology research

Rüegg

Gries

Bond‐Lamberty

et al. 2014

Frontiers in Ecol & Environ

View full text Add to dashboard Cite

Broad-scale temporal or spatial scientific investigations, such as those represented by macrosystems ecology (MSE) projects, address very complex problems that require the collection and synthesis of data from many sources, the collaboration of people from diverse disciplines, and the application of highly complex analytical approaches (Goring et al. 2014;Heffernan et al. 2014). The thorough and transparent documentation of procedures for data collection, processing, and analysis is critical for the success of such projects, and effective information management strategies are required. A wide range of approaches to information management are currently in use, from modest informal information management by individual investigators, to one or more information managers supporting a multi-investigator project (eg a Long Term Ecological Research [LTER] site), to an entire information technology department supporting research platforms (eg National Ecological Observatory Network [NEON]). Most MSE projects fall somewhere on the continuum between the extremes of a single investigator and a NEON-type platform in their information management needs, protocols, and procedures.Data are valuable beyond the original MSE project and should be preserved and made accessible, particularly if public funds were used in their creation (eg National Science Foundation [NSF]). Time, effort, and potentially expensive equipment are needed to collect data that, in a changing world, quickly become irreplaceable (Wolkovich et al. 2012), and many MSE projects rely on previously collected data. However, publishing data requires offering other researchers and the public unfettered and full access to those data (Molloy 2011). For the researcher this means relinquishing complete control over one's data, as well as exposing the data and research to a greater degree of scrutiny than in the past. This prospect, and the reluctance felt by some researchers regarding "open science", is as old as scientific discoveries themselves. The advent of scientific journals facilitated an openness with regard to information, as long as all the data and procedures could be published in a journal article (Nielsen 2012). Yet contemporary science has long surpassed the ability to include all the data in journal articles, and com- MACROSYSTEMS ECOLOGYCompleting the data life cycle: using information management in macrosystems ecology research An important goal of macrosystems ecology (MSE) research is to advance understanding of ecological systems at both fine and broad temporal and spatial scales. Our premise in this paper is that MSE projects require integrated information management at their inception. Such efforts will lead to improved communication and sharing of knowledge among diverse project participants, better science outcomes, and more transparent and accessible (ie "open") science. We encourage researchers to "complete the data life cycle" by publishing welldocumented datasets, thereby facilitating re-use of the data to answer new and different questions f...

show abstract

Section: Data Collection: Incorporating Information Management Earlymentioning

confidence: 99%

Section: Macrosystems Ecologymentioning

confidence: 99%

Completing the data life cycle: using information management in macrosystems ecology research

Rüegg

Gries

Bond‐Lamberty

et al. 2014

Frontiers in Ecol & Environ

View full text Add to dashboard Cite

show abstract

“…Meanwhile, several middleware solutions were proposed to transform streaming data into RDF streams using Semantic Web tools. For example, DataTurbine [21] engine is a streaming data middleware that follows the "publish-subscribe" model to deliver the data in near real-time from sensors to the Cloud for later analysis. Another middleware solution called Linked Stream Middleware [25] was introduced to transform the raw sensory data into RDF streams using W3C's Semantic Sensor Networks Incubator Group [32] ontology.…”

Section: Rdf Stream Processingmentioning

confidence: 99%

Transforming Big Data Into Knowledge Using Semantic Stream Processing Technology: Challenges and Early Progress

Zălhan¹

2019

Proceedings of the 18th International Conference on INFORMATICS in ECONOMY Education, Research and Business Technologies

View full text Add to dashboard Cite

In the Big Data era, the volume of streaming data generated from heterogeneous data sources such as social media, smart devices or sensor networks is overwhelming but enables human users and applications to better understand the world around us. Understanding the context, situation or the environment where the data has been produced is a challenging task due the dynamic and fast nature of the streaming data. Modelling the big data using RDF model and applying continuous SPARQL queries over it using Semantic Web tools helps create machine-understandable meanings and valuable insights of the underlying data. This paper provides an overview of the Big Data theoretical foundation, challenges and stages that need to be taken into consideration when building a Big Data pipeline architecture. Also, the research efforts that have been made in the past years using Semantic Stream Processing techniques are described.

show abstract

“…All tests were shared in real time both with project partners (University of Roma Tre and University of Miami) and with researchers and experts working in the field of seismic protection in a remote configuration, by structural dynamic, numerical simulation, qualification tests and vibration control (DySCo) virtual laboratory via Adobe Connect (De Canio et al, 2013) and by E3S architecture via RBNB DataTurbine (DT) technology (http://dataturbine.org; http://dataturbine. org/documentation/setting-up-a-sink/real-time-dataviewer-rdv/) for near real-time data managing and visualisation (Fountain et al, 2012).…”

Section: Case Studymentioning

confidence: 99%

From 2D digital imaging to finite element analysis using the ENEAGRID high performance computing infrastructure for the preservation of historical masonry structures

Mongelli

Bellagamba

Iannone

et al. 2018

IJMRI

View full text Add to dashboard Cite

To improve the collaboration between scientific communities working within geographically distributed laboratories, it is increasingly necessary to take into account the complexity of data streaming process and the difficulties with integrating heterogeneous instruments of different groups of researchers (Mongelli et al., 2017). This paper shows the 'ENEA staging storage sharing' (E3S) architecture developed within the COBRA project (http://cobra.enea.it/english) which promotes the technology transfer to small and medium-sized companies for the conservation of cultural heritage. This architecture was setup for the shaking tables at the Sustainable Innovation Technologies (SITEC) ENEA Casaccia R.C. Laboratory and it was used during experimental tests on two masonry walls made up of stone and tuff blocks that represent the poor Italian masonry of historical buildings, to verify the behaviour of innovative techniques of anti-seismic reinforcement. E3S supports the whole experimental process, from photogrammetric reconstruction to FEM calibration, reducing time needed to obtain experimental results by means real time data streaming functions.Reference to this paper should be made as follows: Mongelli, M., Bellagamba, I., Iannone, F. and Bracco, G. (2018) 'From 2D digital imaging to finite element analysis using the ENEAGRID high performance computing infrastructure for the preservation of historical masonry structures', Int.

show abstract

The Open Source DataTurbine Initiative: Empowering the Scientific Community with Streaming Data Middleware

Cited by 10 publications

References 11 publications

Completing the data life cycle: using information management in macrosystems ecology research

Completing the data life cycle: using information management in macrosystems ecology research

Transforming Big Data Into Knowledge Using Semantic Stream Processing Technology: Challenges and Early Progress

From 2D digital imaging to finite element analysis using the ENEAGRID high performance computing infrastructure for the preservation of historical masonry structures

Contact Info

Product

Resources

About