Optimising NGAS for the MWA Archive

Wu, Chen; Wicenec, A.; Pallot, D.; Checcucci, A.

doi:10.1007/s10686-013-9354-1

Cited by 17 publications

(13 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CSIRO data centre runs the CSIRO data access portal, an enterprise-wide system that archives and provides access to data across many areas of CSIRO research, including ASKAP/CASDA. The ASKAP data are deposited on to tape and disk storage at Pawsey, with storage managed by the next-generation archive system (NGAS) (Wu et al 2013). Functions such as metadata search and authentication are implemented at the Canberra data centre, while functions that need to be close to the data, such as data deposit and data access, are implemented at Pawsey.…”

Section: Data Archiving and Accessmentioning

confidence: 99%

Australian square kilometre array pathfinder: I. system description

Hotan

Bunton

Chippendale

et al. 2021

Publ. Astron. Soc. Aust.

235

159

View full text Add to dashboard Cite

In this paper, we describe the system design and capabilities of the Australian Square Kilometre Array Pathfinder (ASKAP) radio telescope at the conclusion of its construction project and commencement of science operations. ASKAP is one of the first radio telescopes to deploy phased array feed (PAF) technology on a large scale, giving it an instantaneous field of view that covers $31\,\textrm{deg}^{2}$ at $800\,\textrm{MHz}$ . As a two-dimensional array of 36 $\times$ 12 m antennas, with baselines ranging from 22 m to 6 km, ASKAP also has excellent snapshot imaging capability and 10 arcsec resolution. This, combined with 288 MHz of instantaneous bandwidth and a unique third axis of rotation on each antenna, gives ASKAP the capability to create high dynamic range images of large sky areas very quickly. It is an excellent telescope for surveys between 700 and $1800\,\textrm{MHz}$ and is expected to facilitate great advances in our understanding of galaxy formation, cosmology, and radio transients while opening new parameter space for discovery of the unknown.

show abstract

Section: Data Archiving and Accessmentioning

confidence: 99%

Australian square kilometre array pathfinder: I. system description

Hotan

Bunton

Chippendale

et al. 2021

Publ. Astron. Soc. Aust.

235

159

View full text Add to dashboard Cite

show abstract

“…Each accumulation block is topped with a FITS header (Wells, Greisen, & Harten 1981) and transferred to an archiving system where the products are concatenated into larger FITS files with multiple extensions, wherein each integration is a FITS extension (see Appendix D). The archiving system is an instantiation of the Next Generation Archiving System (NGAS) (Wicenec & Knudstrup 2007; Wu et al 2013) and was originally developed to store data from the European Southern Observatory (ESO) telescopes. The heart of the system is a database that links the output products with the metadata describing the observation and that manages the safe transportation and archiving of data.…”

Section: The Mwa Correlator Systemmentioning

confidence: 99%

The Murchison Widefield Array Correlator

Ord

Crosse

Emrich

et al. 2015

Publ. Astron. Soc. Aust.

Self Cite

View full text Add to dashboard Cite

The Murchison Widefield Array is a Square Kilometre Array Precursor. The telescope is located at the Murchison Radioastronomy Observatory in Western Australia. The MWA consists of 4 096 dipoles arranged into 128 dual polarisation aperture arrays forming a connected element interferometer that cross-correlates signals from all 256 inputs. A hybrid approach to the correlation task is employed, with some processing stages being performed by bespoke hardware, based on Field Programmable Gate Arrays, and others by Graphics Processing Units housed in general purpose rack mounted servers. The correlation capability required is approximately 8 tera floating point operations per second. The MWA has commenced operations and the correlator is generating 8.3 TB day −1 of correlation products, that are subsequently transferred 700 km from the MRO to Perth (WA) in real-time for storage and offline processing. In this paper, we outline the correlator design, signal path, and processing elements and present the data format for the internal and external interfaces.

show abstract

“…It was initially developed by the European Southern Observatory (ESO) (Wicenec et al 2001). The SKA precursor facility, Murchison Widefield Array (MWA), uses the NGAS to synchronize the mirrored archive data from MWA to the data centers at the Massachusetts Institute of Technology (MIT), United States, and the Victoria University of Wellington (VUW), New Zealand (Wu et al 2013). In addition, the NGAS is used by the Atacama Large Millimeter/submillimeter Array (ALMA) to synchronize massive amounts of archive data from ALMA to the data centers located in North America, Europe, and the East Asia (Wicenec et al 2010;Stoehr et al 2014).…”

Section: Introductionmentioning

confidence: 99%

Enhanced Remote Astronomical Archive System Based on the File-Level Unlimited Sliding-Window Technique

Shi,

Deng,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Data archiving is one of the most critical issues for modern astronomical observations. With the development of a new generation of radio telescopes, the transfer and archiving of massive remote data have become urgent problems to be solved. Herein, we present a practical and robust file-level flowcontrol approach, called the Unlimited Sliding-Window (USW), by referring to the classic flow-control method in TCP protocol. Basing on the USW and the Next Generation Archive System (NGAS) developed for the Murchison Widefield Array telescope, we further implemented an enhanced archive system (ENGAS) using ZeroMQ middleware. The ENGAS substantially improves the transfer performance and ensures the integrity of transferred files. In the tests, the ENGAS is approximately three to twelve times faster than the NGAS and can fully utilize the bandwidth of network links. Thus, for archiving radio observation data, the ENGAS reduces the communication time, improves the bandwidth utilization, and solves the remote synchronous archiving of data from observatories such as Mingantu spectral radioheliograph. It also provides a better reference for the future construction of the Square Kilometer Array (SKA) Science Regional Center.

show abstract

Optimising NGAS for the MWA Archive

Cited by 17 publications

References 10 publications

Australian square kilometre array pathfinder: I. system description

Australian square kilometre array pathfinder: I. system description

The Murchison Widefield Array Correlator

Enhanced Remote Astronomical Archive System Based on the File-Level Unlimited Sliding-Window Technique

Contact Info

Product

Resources

About