Big data, but are we ready?

Trelles, Oswaldo; Prins, Pjotr; Snir, Marc; Jansen, Ritsert C.

doi:10.1038/nrg2857-c1

Cited by 135 publications

(82 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Based on the current throughput (,20 MB s 21 ) of Blu-ray discs, it might take more than one and a half years to access 1 PB of information. 3 Even taking the current HDD technique into consideration (,100 MB s 21 ), it would still require more than 100 days to access such a massive amount of data. Therefore, it is equally as Optical storage arrays M Gu et al 4 important to boost writing/reading throughputs as it is to increase storage capacity.…”

Section: Ultrahigh Throughputsmentioning

confidence: 99%

“…Big data centers with petabyte (PB) and even exabyte (EB) capacities have emerged as the key enabling techniques and core platforms for cloud computing and storage, which have been heralded as the primary technical solution for the next generation of big data storage. 2,3 However, the infrastructure of current information technology has been increasingly challenged by the limited storage capacity of current digital media as well as the shortcomings associated with the high energy consumption and short lifetime of such media.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optical storage arrays: a perspective for future big data storage

2014

View full text Add to dashboard Cite

The advance of nanophotonics has provided a variety of avenues for light-matter interaction at the nanometer scale through the enriched mechanisms for physical and chemical reactions induced by nanometer-confined optical probes in nanocomposite materials. These emerging nanophotonic devices and materials have enabled researchers to develop disruptive methods of tremendously increasing the storage capacity of current optical memory. In this paper, we present a review of the recent advancements in nanophotonics-enabled optical storage techniques. Particularly, we offer our perspective of using them as optical storage arrays for next-generation exabyte data centers.

show abstract

Section: Ultrahigh Throughputsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Optical storage arrays: a perspective for future big data storage

2014

View full text Add to dashboard Cite

show abstract

“…In ecology and related environmental sciences, datasets are growing in size, complexity, and type as a result of technological advances in sensor and sensor platform technologies (space-, air-, land-, aquatic-, marine-, and organismal-based), computational and analytical improvements in simulation models, and improved methodologies for probing samples, such as genome sequencing and the generation of 'omics' data (Drake et al 2006, Hart and Martinez 2006, Cohen et al 2009, Luo et al 2011, Pfeifer et al 2012, Porter et al 2012). Research and development into using this data deluge have focused on cyber-infrastructure (CI) hardware and software constraints, the discovery and access to ''dark data'' and ''deep web'' information, and cultural concerns about sharing data (Price and Sherman 2001, Heidorn 2008, Trelles et al 2011, Michener and Jones 2012, Parr et al 2012, Peters et al 2014a) that lead to calls for open science (Wolkovich et al 2012, Hamilton et al 2013. In spite of these advances in data acquisition and publishing, however, the use and re-use of data are not fully exploited.…”

Section: Introductionmentioning

confidence: 99%

Harnessing the power of big data: infusing the scientific method with machine learning to transform ecology

Peters¹,

Havstad²,

Cushing³

et al. 2014

Ecosphere

137

View full text Add to dashboard Cite

Abstract. Most efforts to harness the power of big data for ecology and environmental sciences focus on data and metadata sharing, standardization, and accuracy. However, many scientists have not accepted the data deluge as an integral part of their research because the current scientific method is not scalable to large, complex datasets. Here, we explain how integrating a data-intensive, machine learning approach with a hypothesis-driven, mechanistic approach can lead to a novel knowledge, learning, analysis system (KLAS) for discovery and problem solving. Machine learning leads to more efficient, user-friendly analytics as the streams of data increase while hypothesis-driven decisions lead to the strategic design of experiments to fill knowledge gaps and to elucidate mechanisms. KLAS will transform ecology and environmental sciences by shortening the time lag between individual discoveries and leaps in knowledge by the scientific community, and will lead to paradigm shifts predicated on open access data and analytics in a machine learning environment.

show abstract

“…Full-text search refers to techniques for searching a single formatted document or a collection in a full text database, which is distinguished from searches based on metadata or on parts of the original texts represented in databases [2]. However, the quantity of the hierarchical documents has become bottlenecks on the full-text searching in this context of big data [3]. For instance, how to full-text search millions of hierarchical publications with Digital Object Identifier (DOI) is challenging.…”

mentioning

confidence: 99%

Toward full-text searching middleware over hierarchical documents

Yang

Abraham

2013

2013 13th International Conference on Intellient Systems Design and Applications

View full text Add to dashboard Cite

Abstract-Currently, full-text searching can benefit from the emerging NoSQL databases and traditional indexing tools in the big data era. However, there are some drawbacks of current solutions. On one hand, the indexing documents lack of the hierarchy. On the other hand, big data have become the bottleneck of fulltext searching. In the context of big data, we design a full-text searching middleware over hierarchical documents. We discuss the architecture of this middleware in detail. In addition, we propose a structure-independent hierarchical document model to present the hierarchical document. Moreover, the transformation engine is designed to translate the rich files into models. The core log event listener is responsible for capturing the changed documents and push them to the indexing storage at the same time. The experimental results show that our middleware is more advantageous than RDBMS with indexes and RDBMS with Lucene solutions.

show abstract

Big data, but are we ready?

Cited by 135 publications

References 9 publications

Optical storage arrays: a perspective for future big data storage

Optical storage arrays: a perspective for future big data storage

Harnessing the power of big data: infusing the scientific method with machine learning to transform ecology

Toward full-text searching middleware over hierarchical documents

Contact Info

Product

Resources

About