Resource Harvesting within the OAI-PMH Framework

Sompel, Herbert Van de; Nelson, Michael L.; Lagoze, Carl; Warner, Simeon

doi:10.1045/december2004-vandesompel

Cited by 80 publications

(54 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Several technical standards developed by the digital library community will underpin distributed access to scientific data, documents, and composite objects: Reference Model for Open Archival Information Systems (OAIS), Open Archives Initiative Protocols for Metadata Harvesting (OAI-PMH), OpenURL, the Info URI scheme, and Object Reuse and Exchange (ORE) (Bekaert & Van de Sompel, 2006;Chudnov et al, 2005; Document Action: 'The "info" URI Scheme for Information Assets with Identifiers in Public Namespaces' to Informational RFC, 2005;Object Reuse and Exchange, 2006;Van de Sompel et al, 2004). OAI-PMH facilitates the creation of a platform-independent content layer to support discovery services.…”

Section: Digital Libraries For Datamentioning

confidence: 99%

Little science confronts the data deluge: habitat ecology, embedded sensor networks, and digital libraries

Borgman

Wallis

Enyedy³

2007

Int J Digit Libr

167

157

View full text Add to dashboard Cite

Abstracte-Science promises to increase the pace of science via fast, distributed access to computational resources, analytical tools, and digital libraries. "Big science" fields such as physics and astronomy that collaborate around expensive instrumentation have constructed shared digital libraries to manage their data and documents, while "little science" research areas that gather data through hand-crafted fieldwork continue to manage their data locally. As habitat ecology researchers begin to deploy embedded sensor networks, they are confronting an array of challenges in capturing, organizing, and managing large amounts of data. The scientists and their partners in computer science and engineering make use of common datasets but interpret the data differently. Studies of this field in transition offer insights into the role of digital libraries in e-Science, how data practices evolve as science becomes more instrumented, and how scientists, computer scientists, and engineers collaborate around data. Among the lessons learned are that data on the same variables are gathered by multiple means, that data exist in many states and in many places, and that publication practices often drive data collection practices. Data sharing is embraced in principle but little sharing actually occurs, due to interrelated factors such as lack of demand, lack of standards, and concerns about publication, ownership, data quality, and ethics. We explore the implications of these findings for data policy and digital library architecture. Research reported here is affiliated with the Center for Embedded Networked Sensing.

show abstract

Section: Digital Libraries For Datamentioning

confidence: 99%

Little science confronts the data deluge: habitat ecology, embedded sensor networks, and digital libraries

Borgman

Wallis

Enyedy³

2007

Int J Digit Libr

167

157

View full text Add to dashboard Cite

show abstract

“…In contrast, the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH, see Van de Sompel et al, 2004) 12 is a simple to use, HTTP-based protocol that even supports incremental harvesting. The protocol is widely used within the library world.…”

Section: Protocols For Metadata Retrievalmentioning

confidence: 99%

Generic XML-based framework for metadata portals

Schindler

Diepenbroek

2008

Computers & Geosciences

View full text Add to dashboard Cite

We present a generic and flexible framework for building geoscientific metadata portals independent of content standards for metadata and protocols. Data can be harvested with commonly used protocols (e.g., Open Archives Initiative Protocol for Metadata Harvesting) and metadata standards like DIF or ISO 19115. The new Java-based portal software supports any XML encoding and makes metadata searchable through Apache Lucene. Software administrators are free to define searchable fields independent of their type using XPath. In addition, by extending the fulltext search engine (FTS) Apache Lucene, we have significantly improved queries for numerical and date/time ranges by supplying a new trie-based algorithm, thus enabling high-performance space/time retrievals in FTS-based geo portals. The harvested metadata are stored in separate indexes, which makes it possible to combine these into different portals. The portal-specific Java API and web service interface is highly flexible and supports custom front-ends for users, provides automatic query completion (AJAX), and dynamic visualization with conventional mapping tools. The software has been made freely available through the open source concept.

show abstract

“…Introducing complex object formats as metadata formats in the OAI-PMH framework yields a robust and general solution to the resource-harvesting problem [33]. Especially, unlike other approaches aimed at gathering resources (not just metadata) based on OAI-PMH harvesting, an approach based on complex object representations of assets guarantees that a change to any constituent of a resource will result in a change of the OAI-PMH datestamp of the complex object representation of the resource.…”

Section: Xml Signatures For Constituent Datastreams Of An Assetmentioning

confidence: 99%

A Standards-based Solution for the Accurate Transfer of Digital Assets

Bekaert

Sompel²

2005

D-Lib Magazine

Self Cite

View full text Add to dashboard Cite

This article describes results of a collaboration between the Research Library of the Los Alamos National Laboratory (LANL) and the American Physical Society (APS) aimed at designing and implementing a robust solution for the recurrent transfer of digital assets from the APS collection to LANL. In this solution, various recent standards are combined to obtain an asset transfer framework that should be attractive as a means to optimize content transfer in environments beyond the specific APS/LANL project. The proposed solution uses an XML-based complex object format (the MPEG-21 Digital Item Declaration Language) for the application-neutral representation of compound digital assets of all sorts. It uses a pull-oriented HTTP-based protocol (the Open Archives Initiative Protocol for Metadata Harvesting) that allows incrementally collecting new and updated assets, represented as XML documents, from a producing archive. It builds on an XML-specific technique (W3C XML Signatures) to provide guarantees regarding authenticity and accuracy of the transferred assets.

show abstract

Resource Harvesting within the OAI-PMH Framework

Cited by 80 publications

References 9 publications

Little science confronts the data deluge: habitat ecology, embedded sensor networks, and digital libraries

Little science confronts the data deluge: habitat ecology, embedded sensor networks, and digital libraries

Generic XML-based framework for metadata portals

A Standards-based Solution for the Accurate Transfer of Digital Assets

Contact Info

Product

Resources

About