The Lofar Central Processing Facility Architecture

Abstract

Reconfiguration is a key feature characteristic of the LOFAR telescope. Software platforms are utilised to program out the required data transformations in the generation of scientific end-products. Reconfigurable resources nowadays often replace the hard-wired processing systems from the past. This paper describes how this paradigm is implemented in a purely general-purpose telescope back-end. Experiences from high performance computing, stream processing and software engineering have been combined, leading to a state-of-the-art processing platform. The processing platform offers a total processing power of 35 TFlops, which is used to process a sustained input datastream of 320 Gbps. The architecture of this platform is optimised for streaming data processing and offers appropriate processing resources for each step in the data processing chains. Typical data processing chains include Fourier transformations and correlation tasks along with controlling tasks such as fringe rotation correction. These tasks are defined in a high level programming language and mapped onto the available resources at run time. A scheduling system is used to control a collection of concurrently executing observations, providing each associated application with the appropriate resources to meet its timing constraint and give the integrated system the correct on-line and off-line look and feel.

Astronomical real-time streaming signal processing on a Blue Gene/L supercomputer

“…Our application is built on the CEntral Processing Framework CEPframe [6] software. CEPframe supports streaming-data applications by providing transparent communication between different types of hosts, as well as configuration management, monitoring, and fault tolerance.…”

“…Its design breaks radically with conventional telescopes: rather than using large, expensive dishes, LOFAR is built as a distributed sensor network of simple antenna receivers [1]. Their data are centrally collected and processed on the CEntral Processing facility (CEP), that consists of a Blue Gene/L supercomputer and a number of conventional cluster computers [7,6]. LOFAR is built in a three-level hierarchy.…”

Integrating FTS in the Fenix HPC Infrastructure

“…This applies to various science instruments that are in need of HPC for processing the data. This approach has been pioneered by the radio astronomy community (see, e.g., [2]). In the upcoming High Luminosity phase of the Large Hadron Collider (HL-LHC), the compute requirements of experiments are expected to significantly increase such that HPC resources will be needed (see, e.g., estimates from the Atlas experiment [3]).…”

Data Provenance and Management in Radio Astronomy: A Stream Computing Approach

“…It is estimated that LOFAR with its 36 antennae stations can produce over 100 TB/day [3]. For the SKA which will eventually have about 3000 antennae dishes, the data will increase by at least 5 orders of magnitude [4].…”