Synchrotron Big Data Science

Abstract

The rapid development of synchrotrons has massively increased the speed at which experiments can be performed, while new techniques have increased the amount of raw data collected during each experiment. While this has created enormous new opportunities, it has also created tremendous challenges for national facilities and users. With the huge increase in data volume, the manual analysis of data is no longer possible. As a result, only a fraction of the data collected during the time- and money-expensive synchrotron beam-time is analyzed and used to deliver new science. Additionally, the lack of an appropriate data analysis environment limits the realization of experiments that generate a large amount of data in a very short period of time. The current lack of automated data analysis pipelines prevents the fine-tuning of beam-time experiments, further reducing their potential usage. These effects, collectively known as the "data deluge," affect synchrotrons in several different ways including fast data collection, available local storage, data management systems, and curation of the data. This review highlights the Big Data strategies adopted nowadays at synchrotrons, documenting this novel and promising hybridization between science and technology, which promise a dramatic increase in the number of scientific discoveries.

The evolving landscape for alloy design

“…Many synchrotron beamlines generate on the orders of 1 TB/day of data; advanced detectors such as the GigaFRoST detector at the Swiss Light Source now generate 8.8 GB/s. 106 The rate-limiting step is no longer acquisition of materials information, but reconstruction and analysis of that information. Collaborative efforts across beamline facilities are underway to deploy massively parallel codes and real-time algorithms.…”

Quantum Dots Microstructural Metrology: From Time‐Resolved Spectroscopy to Spatially Resolved Electron Microscopy

“…TR X-ray micro scopy generates multidimensional data corresponding to 3D real space, 3D reciprocal space, and multidimensional spectra, and the analysis of these data awakes the necessity of progress in the computer vision and image reconstruction. [168,169] Single-molecule absorption-scanning tunneling microscopy (SMA-STM) is an emerging tool to image the electronic excited states at different orientation that could give a better 3D structure view of the excited states. [170] The SMA-STM is advancing towards the possibility of single-particle tomography of excited states that could highlight the defects rather than an average structure obtained via cryo-TEM.…”

Deep denoising for multi-dimensional synchrotron X-ray tomography without high-quality reference data

“…Sub-pixel inaccuracies may cause blurring of the network output, for example. Finally, with fewer than 50 synchrotron facilities worldwide, beamtime is a scarce resource 16 , and acquiring additional measurements may simply be too expensive.…”