Square Kilometre Array : Processing Voluminous MeerKAT Data on IRIS

Abstract

Processing astronomical data often comes with huge challenges with regards to data management as well as data processing. MeerKAT telescope is one of the precursor telescopes of the World's largest observatory Square Kilometre Array. So far, MeerKAT data was processed using the South African computing facility i.e. IDIA, and exploited to make ground-breaking discoveries. However, to process MeerKAT data on UK's IRIS computing facility requires new implementation of the MeerKAT pipeline. This paper focuses on how to transfer MeerKAT data from the South African site to UK's IRIS systems for processing. We discuss about our RapifXfer Data transfer framework for transferring the MeerKAT data from South Africa to the UK, and the MeerKAT job processing framework pertaining to the UK's IRIS resources.

DUG Insight: A software package for big-data analysis and visualisation, and its demonstration for passive radar space situational awareness using radio telescopes

“…For example, CARTA (Cube Analysis and Rendering Tool for Astronomy) is an analysis tool designed for image visualisation and analysis at the scale of SKA precursor telescopes, with the aim of being scalable for future telescopes (Comrie et al, 2021). Large-scale data transfer between MeerKAT, an SKA precursor in South Africa (Booth and Jonas, 2012) and the UK, has been explored to utilise processing capacity at the IRIS (Incorporated Research Institutions for Seismology) computing facilities (Thavasimani and Scaife, 2021).…”

Gradient-Guided Attentional Network for Radio Transient Localization With the Cluster-Feed Telescope

“…directly proportional to the baseline length [3], [4]. As an interferometer used for astronomical observations, it requires a group of reflectors and feeds elements, which generates massive data streams [5]. It poses many challenges in terms of real-time data processing.…”