A Framework for Evaluation of Composite Memento Temporal Coherence

Ainsworth, Scott G.; Nelson, Michael L.; Sompel, Herbert Van de

doi:10.48550/arxiv.1402.0928

Cited by 6 publications

(8 citation statements)

References 6 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used the Squidwarc headless crawler [11] to load each URI-M (including executing JavaScript to ensure loading all embedded resources) and download the contents into a WARC file [16]. Saving the data in WARC files allowed us to record all HTTP response headers and content for all of the resources that made up the composite memento [1].…”

Section: Methodsmentioning

confidence: 99%

“…Web archives are established with the objective of providing permanent access to archived web pages, or mementos. 1 Mementos should be accessible in web archives even after the corresponding live web page is no longer available. The Uniform Resource Identifier (URI) [12] of the archived web page should not change over time, otherwise this defeats the purpose of using archived URIs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Where Did the Web Archive Go?

Aturban¹,

Nelson²,

Weigle³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

To perform a longitudinal investigation of web archives and detecting variations and changes replaying individual archived pages, or mementos, we created a sample of 16,627 mementos from 17 public web archives. Over the course of our 14-month study (November, 2017 -January, 2019), we found that four web archives changed their base URIs and did not leave a machine-readable method of locating their new base URIs, necessitating manual rediscovery. Of the 1,981 mementos in our sample from these four web archives, 537 were impacted: 517 mementos were rediscovered but with changes in their time of archiving (or Memento-Datetime), HTTP status code, or the string comprising their original URI (or URI-R), and 20 of the mementos could not be found at all.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Where Did the Web Archive Go?

Aturban¹,

Nelson²,

Weigle³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A composite memento refers to all embedded resources that comprise a memento [4]. We modified the shell script (see Figure 8) written by Gwern Branwen [5].…”

Section: Generating a Hash Of A Composite Mementomentioning

confidence: 99%

Difficulties of Timestamping Archived Web Pages

Aturban¹,

Nelson²,

Weigle³

2017

Preprint

Self Cite

View full text Add to dashboard Cite

We show that state-of-the-art services for creating trusted timestamps in blockchain-based networks do not adequately allow for timestamping of web pages. They accept data by value (e.g., images and text), but not by reference (e.g., URIs of web pages). Also, we discuss difficulties in repeatedly generating the same cryptographic hash value of an archived web page. We then introduce several requirements to be fulfilled in order to produce repeatable hash values for archived web pages.

show abstract

“…When the representation is replayed from the archive, the JavaScript will execute and may issue Ajax requests for a resource that is on the live web, which leads to one of two possible outcomes: the live web "leaking" into the archive leading to an incorrect representation [13], or missing embedded resources (i.e., returns a 400 or 500 class HTTP response) in the archived resource leading to an incomplete representation, both of which result in reduced archival quality [10]. When an archived deferred representation loads embedded resources from the live web via leakage, it is a zombie resource, leaving the representation incorrect, and potentially prima facie violative [2]. We refer to the ease of archiving a Web resource as archivability, and have shown that resources that rely on JavaScript to construct their representations have lower archivability than resources that avoid JavaScript [12].…”

Section: Problem Descriptionmentioning

confidence: 99%

Adapting the Hypercube Model to Archive Deferred Representations and Their Descendants

Brunelle¹,

Weigle²,

Nelson³

2016

Preprint

Self Cite

View full text Add to dashboard Cite

The web is today's primary publication medium, making web archiving an important activity for historical and analytical purposes. Web pages are increasingly interactive, resulting in pages that are increasingly difficult to archive. Client-side technologies (e.g., JavaScript) enable interactions that can potentially change the client-side state of a representation. We refer to representations that load embedded resources via JavaScript as deferred representations. It is difficult to archive all of the resources in deferred representations and the result is archives with web pages that are either incomplete or that erroneously load embedded resources from the live web.We propose a method of discovering and crawling deferred representations and their descendants (representation states that are only reachable through client-side events). We adapt the Dincturk et al. Hypercube model to construct a model for archiving descendants, and we measure the number of descendants and requisite embedded resources discovered in a proof-of-concept crawl. Our approach identified an average of 38.5 descendants per seed URI crawled, 70.9% of which are reached through an onclick event. This approach also added 15.6 times more embedded resources than Heritrix to the crawl frontier, but at a rate that was 38.9 times slower than simply using Heritrix. We show that our dataset has two levels of descendants. We conclude with proposed crawl policies and an analysis of the storage requirements for archiving descendants.

show abstract

A Framework for Evaluation of Composite Memento Temporal Coherence

Cited by 6 publications

References 6 publications

Where Did the Web Archive Go?

Where Did the Web Archive Go?

Difficulties of Timestamping Archived Web Pages

Adapting the Hypercube Model to Archive Deferred Representations and Their Descendants

Contact Info

Product

Resources

About