MemGator - A Portable Concurrent Memento Aggregator

Alam, Sawood; Nelson, Michael L.

doi:10.1145/2910896.2925452

Cited by 27 publications

(11 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, we deployed MemGator to determine if copies of the websites were also present in web archives around the world. MemGator is able to search the web archives of many institutions and return a list of archived copies of a website, the time and date they were captured, and the institution that captured them (Alam & Nelson, 2016). Table 2 Although 13% -46% of the websites from the Western Canadian heritage collections disappeared from the live web, it is promising to see the thousands of captures by the University of Alberta (using Archive-It) and the Internet Archive.…”

Section: Methodology and Discussionmentioning

confidence: 99%

Assessing the loss of Western Canadian digital heritage

Saiyera

Ayala

2021

CAIS/ACSI

View full text Add to dashboard Cite

In 2003, UNESCO recognized the volatility of our heritage on the web. In response, many Canadian institutions rose to the challenge to preserve our digital heritage. This study examines web archives created by the University of Alberta Libraries relevant to Western Canadian heritage. We examine these collections in order to (1) assess their degree of link rot (which occurs when a website is no longer online) and (2) to determine how extensively these websites have been preserved.

show abstract

Section: Methodology and Discussionmentioning

confidence: 99%

Assessing the loss of Western Canadian digital heritage

Saiyera

Ayala

2021

CAIS/ACSI

View full text Add to dashboard Cite

show abstract

“…This technique of chaining the output as input to the next step reduced the generation time for subsequent MementoMaps from hours to a few minutes and also illustrated that MementoMaps can easily be compacted further when needed. 7 https://gist.github.com/ibnesayeed/bb167fe19c5719d87c1c1f665001d44b 8 https://gist.github.com/ibnesayeed/7307f0bf1783357db99f8b2357249dd0 Figure 9: Growth of compacted MementoMap vs. lines processed from an input MementoMap. This plot illustrates a very small portion of the entire process to highlight the compaction behavior at a micro level.…”

Section: Mementomap Cost and Accuracymentioning

confidence: 99%

MementoMap Framework for Flexible and Adaptive Web Archive Profiling

Alam

Weigle

Nelson

et al. 2019

2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL)

Self Cite

View full text Add to dashboard Cite

In this work we propose MementoMap, a flexible and adaptive framework to efficiently summarize holdings of a web archive. We described a simple, yet extensible, file format suitable for MementoMap. We used the complete index of the Arquivo.pt comprising 5B mementos (archived web pages/files) to understand the nature and shape of its holdings. We generated MementoMaps with varying amount of detail from its HTML pages that have an HTTP status code of 200 OK. Additionally, we designed a single-pass, memoryefficient, and parallelization-friendly algorithm to compact a large MementoMap into a small one and an in-file binary search method for efficient lookup. We analyzed more than three years of Mem-Gator (a Memento aggregator) logs to understand the response behavior of 14 public web archives. We evaluated MementoMaps by measuring their Accuracy using 3.3M unique URIs from MemGator logs. We found that a MementoMap of less than 1.5% Relative Cost (as compared to the comprehensive listing of all the unique original URIs) can correctly identify the presence or absence of 60% of the lookup URIs in the corresponding archive while maintaining 100% Recall (i.e., zero false negatives).

show abstract

“…The MemGator software is available for download and local installation (Alam and Nelson, 2016), making it possible to set up a Memento aggregator with custom sets of web archives, suitable for local or private deployments.…”

Section: Memento and Aggregating Multiple Archivesmentioning

confidence: 99%

Adding the Dimension of Time to HTTP

Nelson¹,

Sompel²

2019

The SAGE Handbook of Web History

Self Cite

View full text Add to dashboard Cite

IntroductIon While the web is distributed, most web archives are centralized silos that do not cooperate with each other. This is partially because the technology that is necessary to replay the archived content and keep it from being influenced by material on the live web also makes it difficult for web archives to cooperate. The Memento Protocol (which we played a central role in defining) addresses this problem by defining an extension to the Hypertext Transfer Protocol (HTTP) that allows for standardized, machine-readable integration of both the past web and the present web. The Memento Protocol extends the concept of HTTP content negotiation to include not only well-known dimensions such as Multipurpose Internet Mail Extensions (MIME) types (e.g., JPEG vs. PNG) and file encodings (e.g., gzip vs. compress), but also the dimension of Coordinated Universal Time (UTC) as a universal versioning system. The protocol can be supported by all systems that hold temporal 14 BK-SAGE-BRUGGER_MILLIGAN-180264-Chp14.indd 189

show abstract

MemGator - A Portable Concurrent Memento Aggregator

Cited by 27 publications

References 2 publications

Assessing the loss of Western Canadian digital heritage

Assessing the loss of Western Canadian digital heritage

MementoMap Framework for Flexible and Adaptive Web Archive Profiling

Adding the Dimension of Time to HTTP

Contact Info

Product

Resources

About