Not all mementos are created equal: Measuring the impact of missing resources

Abstract

Web archives do not capture every resource on every page that they attempt to archive. This results in archived pages missing a portion of their embedded resources. These embedded resources have varying historic, utility, and importance values. The proportion of missing embedded resources does not provide an accurate measure of their impact on the Web page; some embedded resources are more important to the utility of a page than others. We propose a method to measure the relative value of embedded resources and assign a damage rating to archived pages as a way to evaluate archival success. In this paper, we show that Web users' perceptions of damage are not accurately estimated by the proportion of missing embedded resources. The proportion of missing embedded resources is a less accurate estimate of resource damage than a random selection. We propose a damage rating algorithm that provides closer alignment to Web user perception, providing an overall improved agreement with users on memento damage by 17% and an improvement by 51% if the mementos are not similarly damaged. We use our algorithm to measure damage in the Internet Archive, showing that it is getting better at mitigating damage over time (going from 0.16 in 1998 to 0.13 in 2013). However, we show that a greater number of important embedded resources (2.05 per memento on average) are missing over time.

The Memento Tracer Framework: Balancing Quality and Scalability for Web Archiving

“…Brunelle et al [6] proposed an automated method to assess the archived quality of web resources. Their algorithm assigns relative values to embedded resources and depending on the availability of these resources, determines a damage rating.…”

Adding the Dimension of Time to HTTP

“…Although the gold standard for assessing web archiving quality is still human interaction with a Memento to ensure all embedded resources, links, and functionality are preserved, this level of assessment is clearly not scalable. We have been involved with a range of automated evaluations of the web archiving process, including the Archival Acid Test (Kelly et al, 2014a), which evaluates the capabilities of crawling and playback technology stacks (e.g., the Heritrix crawler (Mohr et al, 2004) and the Wayback Machine playback engine), and assessing Memento damage (Brunelle et al, 2014(Brunelle et al, , 2015 which provides weights to missing embedded resources based on heuristics for determining if the missing resource was 'important'.…”

A Framework for Aggregating Private and Public Web Archives

“…Adding the ability for this derived attribute to be present in a TimeMap would allow for more efficient evaluation of memento quality. Brunelle et al [7] developed a metric for determining the quality of a capture (cf. quantity in Section 4.1) when dereferencing a URI-M with a particular focus on the quantitative significance of missing embedded resources.…”

“…With the introduction of derived attributes (Section 4.2), it is critical to not just give context as to the semantics of new attributes like "damage" but also to provide guidance in generating this value. Figure 7 provides an example where a derived attribute requiring calculation (memento damage [7]) and an access attribute are defined for guidance within the TimeMap. Definitions in the context utilize a URI to associate semantics of an attribute for a memento.…”