Enhancing Hadoop System Dependability Through Autonomous Snapshot

Big Questions: Digital Preservation of Big Data in Government

“…Apache Hadoop, a cloud computing platform that supports Big Data processing, includes the Hadoop Distributed File System (HDFS), which can take snapshots of partial or entire file systems. 45 Researchers at the University of Applied Sciences Eastern Switzerland HTW Chur developed a prototype for a mixed mode data repository (MMRepo) using Hadoop 2.0 and HDFS2. 46 Ingo Barkow, Catharina Wasner, and Fabian Odoni do not explicitly address the HDFS2 snapshot capability, which normally requires user involvement for a snapshot to be taken.…”

“…However, Tsozen Yeh and Yipin Wang sought to address this limitation by "revis[ing] and improv[ing] the snapshot scheme" so that "when changes [are] made to a part of the file system with snapshots taken before, [the] system will autonomously take a new snapshot on that part of the file system in real time." 47 Yeh and Wang conceptualize the automatic snapshot in terms of restoring backups after errors, rather than preserving authentic records. Interdisciplinary collaboration is needed to bridge technical developments with archival theory.…”