Nakshatra: Towards Running Batch Analytics on an Archive

Kathpal, Atish; Yasa, Giridhar Appaji Nag

doi:10.1109/mascots.2014.67

Cited by 7 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the access patterns, groups of objects that are accessed together are stored on same disk group. Nakshatra [7] is a data processing framework for batch data analytics on tapes using pre-fetching.…”

Section: B Data Processing On Cold Storagementioning

confidence: 99%

Towards Batch-Processing on Cold Storage Devices

Hadian

Heinis

2018

2018 IEEE 34th International Conference on Data Engineering Workshops (ICDEW)

View full text Add to dashboard Cite

Large amounts of data in storage systems is cold, i.e., Written Once and Read Occasionally (WORO). The rapid growth of massive-scale archival and historical data increases the demand for petabyte-scale cheap storage for such cold data. A Cold Storage Device (CSD) is a disk-based storage system which is designed to trade off performance for cost and power efficiency. Inevitably, the design restrictions used in CSD's results in performance limitations. These limitations are not a concern for WORO workloads, however, the very low price/performance characteristics of CSDs makes them interesting for other applications, e.g., batch processes, too. Applications, however, can be very slow on CSD's if they do not take their characteristics into account. In this paper we design two strategies for data partitioning in CSDs-a crucial operation in many batch analytics tasks like hash-join, near-duplicate detection, and data localization. We show that our strategies can efficiently use CSDs for batch processing of terabyte-scale data by accelerating data partitioning by 3.5x in our experiments.

show abstract

Section: B Data Processing On Cold Storagementioning

confidence: 99%

Towards Batch-Processing on Cold Storage Devices

Hadian

Heinis

2018

2018 IEEE 34th International Conference on Data Engineering Workshops (ICDEW)

View full text Add to dashboard Cite

show abstract

“…However, data center managers can modify the tape scheduling policies to best accommodate different dataset profiles. Standard design guidelines impose that no more than a couple of seconds is allotted to scheduling algorithms, e.g., to enable users to run batch analytics directly on archived data (Kathpal and Yasa 2014). Thus, managers must balance the trade-off between the quality of the schedule, measured as a function of the time between when requests are received and when files first become available, and the computational complexity of finding such a schedule.…”

Section: Introductionmentioning

confidence: 99%

On Approximate Sequencing Policies for Linear Storage Devices

Cardonha¹,

Ciré²,

Real³

2021

Preprint

View full text Add to dashboard Cite

This paper investigates scheduling policies for file retrieval in linear storage devices, such as magnetic tapes. Tapes are the technology of choice for long-term storage in data centers due to their low cost per capacity, reliability, and data security. While scheduling problems associated with data retrieval in tapes are classical, existing works focus on more straightforward heuristic approaches due to limited computational times imposed by standard tape specifications. Our first contribution is a theoretical investigation of three standard policies, presenting their worstcase performance and special cases of practical relevance for which they are optimal. Next, we show that the problem is polynomially solvable via two interleaved recursive models, albeit with high computational complexity. We leverage our previous results to develop two new scheduling policies with constant-ratio performance and low computational cost. Finally, we investigate properties associated with the online variant of the problem, presenting a new constant-factor competitive algorithm. Our numerical analysis on synthetic and real-world tapes from an industry partner provides insights into dataset configurations where each policy is more effective, which is of relevance to data center managers. In particular, our new best-performing policy is practical for large datasets and significantly improves upon standard algorithms in the area.

show abstract

Cheap Data Analytics on Cold Storage

Appuswamy¹

2018

Encyclopedia of Big Data Technologies

View full text Add to dashboard Cite

Nakshatra: Towards Running Batch Analytics on an Archive

Cited by 7 publications

References 5 publications

Towards Batch-Processing on Cold Storage Devices

Towards Batch-Processing on Cold Storage Devices

On Approximate Sequencing Policies for Linear Storage Devices

Cheap Data Analytics on Cold Storage

Contact Info

Product

Resources

About