Large-scale Real-time Data-driven Scientific Applications

Abstract

Abstract-Large-scale real-time data processing is becoming common in many scientific disciplines. But processing large amount of data in real-time is still challenging with existing technology. In last few years, the dynamic data driven approach is becoming people's spotlight due to its potential in reducing data intelligently. Enlighten by this concept, a new data-driven framework for large-scale real-time data analysis is proposed in this work and a scientific application under this framework is given in details. By introducing additional information to data analysis processes, large-scale data processing can be achieved with real-time time constraint.

Enabling Distributed Computing Systems with ElopTM

“…• Web services implementation using gSAOP 7 . The reference implementation is developed in the C++ language and compiled on the Linux platform.…”

“…Computing resources are separately described and encapsulated instead of using a traditional all-in-one model. These include computing hosts (physical or virtual machines [4]), data and storage [7], software packages [11] and displays (remote displays or virtual desktops [16]). To solve the A4 problem, the physical properties of computing elements are totally abstracted and described in a logical way.…”