The common pipeline library: standardizing pipeline processing

McKay, D. J.; Ballester, P.; Banse, Klaus; Izzo, C.; Jung, Yves; Kiesgen, M.; Kornweibel, Nick; Lundin, Lars; Modigliani, A.; Palsa, Ralf; Sabet, Cyrus

doi:10.1117/12.551214

Cited by 10 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The mechanism to identify files to the recipes is different in different environments. For example, IRAF uses different input parameters for different file types, whereas ESO's CPL recipes (McKay et al 2004) use text files with file tags to identify the file types. In both cases, Reflex uses the category to identify these file types.…”

Section: The Reflex Environmentmentioning

confidence: 99%

Automated data reduction workflows for astronomy

Freudling

Romaniello

Bramich

et al. 2013

A&A

642

483

View full text Add to dashboard Cite

Context. Data from complex modern astronomical instruments often consist of a large number of different science and calibration files, and their reduction requires a variety of software tools. The execution chain of the tools represents a complex workflow that needs to be tuned and supervised, often by individual researchers that are not necessarily experts for any specific instrument. Aims. The efficiency of data reduction can be improved by using automatic workflows to organise data and execute a sequence of data reduction steps. To realize such efficiency gains, we designed a system that allows intuitive representation, execution and modification of the data reduction workflow, and has facilities for inspection and interaction with the data. Methods. The European Southern Observatory (ESO) has developed Reflex, an environment to automate data reduction workflows. Reflex is implemented as a package of customized components for the Kepler workflow engine. Kepler provides the graphical user interface to create an executable flowchart-like representation of the data reduction process. Key features of Reflex are a rule-based data organiser, infrastructure to re-use results, thorough book-keeping, data progeny tracking, interactive user interfaces, and a novel concept to exploit information created during data organisation for the workflow execution. Results. Automated workflows can greatly increase the efficiency of astronomical data reduction. In Reflex, workflows can be run noninteractively as a first step. Subsequent optimization can then be carried out while transparently re-using all unchanged intermediate products. We found that such workflows enable the reduction of complex data by non-expert users and minimizes mistakes due to book-keeping errors. Conclusions. Reflex includes novel concepts to increase the efficiency of astronomical data processing. While Reflex is a specific implementation of astronomical scientific workflows within the Kepler workflow engine, the overall design choices and methods can also be applied to other environments for running automated science workflows.

show abstract

Section: The Reflex Environmentmentioning

confidence: 99%

Automated data reduction workflows for astronomy

Freudling

Romaniello

Bramich

et al. 2013

A&A

642

483

View full text Add to dashboard Cite

show abstract

“…Pipeline recipes can be executed either with the ESO recipe execution tool (EsoRex 2 ) at the command line level or through the Gasgano 3 graphical user interface. The recipes are implemented with the ESO Common Pipeline Library (CPL, Banse et al 2004;McKay et al 2004).…”

Section: Data Reduction Softwarementioning

confidence: 99%

X-shooter, the new wide band intermediate resolution spectrograph at the ESO Very Large Telescope

Vernet

Dekker

D’Odorico

et al. 2011

A&A

977

481

View full text Add to dashboard Cite

X-shooter is the first 2nd generation instrument of the ESO Very Large Telescope (VLT). It is a very efficient, single-target, intermediate-resolution spectrograph that was installed at the Cassegrain focus of UT2 in 2009. The instrument covers, in a single exposure, the spectral range from 300 to 2500 nm. It is designed to maximize the sensitivity in this spectral range through dichroic splitting in three arms with optimized optics, coatings, dispersive elements and detectors. It operates at intermediate spectral resolution (R ∼ 4000−17 000, depending on wavelength and slit width) with fixed échelle spectral format (prism cross-dispersers) in the three arms. It includes a 1.8 × 4 integral field unit as an alternative to the 11 long slits. A dedicated data reduction package delivers fully calibrated two-dimensional and extracted spectra over the full wavelength range. We describe the main characteristics of the instrument and present its performance as measured during commissioning, science verification and the first months of science operations.

show abstract

“…The asterisk denotes unnamed data reduction software. The ESO Common Pipeline Library (CPL) is a collection of pipelines for ESO instruments where each instrument has specific scripts and some shared functionality (McKay et al 2004).…”

Section: The Users Of Aperomentioning

confidence: 99%

APERO: A PipelinE to Reduce Observations—Demonstration with SPIRou

Cook¹,

Artigau

Doyon

et al. 2022

PASP

View full text Add to dashboard Cite

With the maturation of near-infrared high-resolution spectroscopy, especially when used for precision radial velocity, data reduction has faced unprecedented challenges in terms of how one goes from raw data to calibrated, extracted, and corrected data with required precisions of thousandths of a pixel. Here we present A PipelinE to Reduce Observations (apero), specifically focused on Spectro Polarimètre Infra ROUge (SPIRou), the near-infrared spectropolarimeter on the Canada–France–Hawaii Telescope (SPectropolarimètre InfraROUge, CFHT). In this paper, we give an overview of apero and detail the reduction procedure for SPIRou. apero delivers telluric-corrected 2D and 1D spectra as well as polarimetry products. apero enables precise stable radial velocity measurements on the sky (via the LBL algorithm), which is good to at least ∼2 m s−1 over the current 5 yr lifetime of SPIRou.

show abstract

The common pipeline library: standardizing pipeline processing

Cited by 10 publications

References 0 publications

Automated data reduction workflows for astronomy

Automated data reduction workflows for astronomy

X-shooter, the new wide band intermediate resolution spectrograph at the ESO Very Large Telescope

APERO: A PipelinE to Reduce Observations—Demonstration with SPIRou

Contact Info

Product

Resources

About