Examining extended and scientific metadata for scalable index designs

Parker-Wood, Aleatha; Long, Darrell D. E.; Madden, Brian A.; Adams, Ian F.; McThrow, Michael; Wildani, Avani

doi:10.1145/2485732.2485754

Cited by 5 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the explosively growing size of data storage systems, some recent studies [16,17] collect and analyze characteristics of various system workloads in practical application. From these studies, some new features of the file metadata are found.…”

Section: File Metadata and Metadata Searchmentioning

confidence: 99%

See 1 more Smart Citation

Using provenance to efficiently improve metadata searching performance in storage systems

Liu

Feng

Hua

et al. 2015

Future Generation Computer Systems

View full text Add to dashboard Cite

Section: File Metadata and Metadata Searchmentioning

confidence: 99%

“…The R-tree has been the most popular spatial index structure, and it is more suitable for indexing multi-dimensional attributes than B-tree [7,17]. X-tree is a variant of the R-tree.…”

Section: Overhead Of Metadata Searchingmentioning

confidence: 99%

Using provenance to efficiently improve metadata searching performance in storage systems

Liu

Feng

Hua

et al. 2015

Future Generation Computer Systems

View full text Add to dashboard Cite

“…As ever increasing volumes of data (which some call the digital landfill) are generated, it becomes nearly impossible to store and retrieve data with a single computer. The management of large volumes of astronomical FITS data files is beyond the scope of traditional data indexing technologies (Greisen et al 1981;Gray et al 2005;Parker-Wood et al 2013). Thus, NoSQL technology was presented, offering a timely alternative to the object-relational data model.…”

Section: Related Workmentioning

confidence: 99%

NVST Data Archiving System Based on Fastbit Nosql Database

Liu¹,

Wei²,

Ji³

et al. 2014

Journal of The Korean Astronomical Society

View full text Add to dashboard Cite

The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution imaging and spectral observations, including the measurements of the solar magnetic field. The NVST has been collecting more than 20 million FITS files since it began routine observations in 2012 and produces a maximum observational records of 120 thousand files in a day. Given the large amount of files, the effective archiving and retrieval of files becomes a critical and urgent problem. In this study, we implement a new data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL) database. Comparing to the relational database (i.e., MySQL; My Structured Query Language), the Fastbit database manifests distinctive advantages on indexing and querying performance. In a large scale database of 40 million records, the multi-field combined query response time of Fastbit database is about 15 times faster and fully meets the requirements of the NVST. Our study brings a new idea for massive astronomical data archiving and would contribute to the design of data management systems for other astronomical telescopes.

show abstract

“…The study of [18] proposes an architecture that uses a column store as a basis for a lazy index. In this process, fields are stored, unprocessed, embedded in the data until the first query is issued.…”

Section: Other Studies On Fast Data Processingmentioning

confidence: 99%

Study on Potential Capabilities of a Nodb System

Y¹,

L²

2014

IJIT

View full text Add to dashboard Cite

There is a need of optimal data to query processing technique to handle the increasing database size, complexity, diversity of use. With the introduction of commercial website, social network, expectations are that the high scalability, more flexible database will replace the RDBMS. Complex application and Big Table require highly optimized queries. Users are facing the increasing bottlenecks in their data analysis. A growing part of the database community recognizes the need for significant and fundamental changes to database design. A new philosophy for creating database systems called noDB aims at minimizing the datato-query time, most prominently by removing the need to load data before launching queries. That will process queries without any data preparation or loading steps. There may not need to store data. User can pipe raw data from websites, DBs, excel sheets into two promise sample inputs without storing anything. This study is based on PostgreSQL systems. A series of the baseline experiment are executed to evaluate the Performance of this system as per-a. Data loading cost, b-Query processing timing, c-Avoidance of Collision and Deadlock, d-Enabling the Big data storage and e-Optimize query processing etc. The study found significant possible capabilities of noDB system over the traditional database management system.

show abstract

Examining extended and scientific metadata for scalable index designs

Cited by 5 publications

References 17 publications

Using provenance to efficiently improve metadata searching performance in storage systems

Using provenance to efficiently improve metadata searching performance in storage systems

NVST Data Archiving System Based on Fastbit Nosql Database

Study on Potential Capabilities of a Nodb System

Contact Info

Product

Resources

About