An open-source software ecosystem for the interactive exploration of ultrafast electron scattering data

Abstract

This paper details a software ecosystem comprising three free and open-source Python packages for processing raw ultrafast electron scattering (UES) data and interactively exploring the processed data. The first package, iris, is graphical user-interface program and library for interactive exploration of UES data. Under the hood, iris makes use of npstreams, an extensions of numpy to streaming array-processing, for high-throughput parallel data reduction. Finally, we present scikit-ued, a library of reusable routines and data structures for analysis of UES data, including specialized image processing algorithms, simulation routines, and crystal structure manipulation operations. In this paper, some of the features or all three packages are highlighted, such as parallel data reduction, image registration, interactive exploration. The packages are fully tested and documented and are released under permissive licenses.

Mechanisms of electron-phonon coupling unraveled in momentum and time: The case of soft phonons in TiSe 2

“…Ultrafast electron scattering data are processed and analyzed using the free and open-source program Iris ( 44 ), which is built on top of the scikit-ued and npstreams Python libraries.…”

An Open Source, Iterative Dual-Tree Wavelet Background Subtraction Method Extended from Automated Diffraction Pattern Analysis to Optical Spectroscopy

“…We were thus excited by a new background subtraction method, the dual-tree complex wavelet transform with an iterative algorithm (DTCWT-IA), that was motivated by the need to cope with large time-varying data sets with appreciable segments of data with no significant range of background-only signal. 22,28 Such a signal characteristic challenges curve-fitting and Fourier methods, and looms forbidding to the prospect of batch processing. Beyond this technical benefit, we were pleased with the practical motivation that the development was directed by a desire to reliably perform background subtractions across large series of measurements with variability in signal and signal-to-noise (S/N) levels, and with minimal user intervention.…”

“…The method combines the earlier promise of iterative wavelet methods applied to Raman spectral analysis 1 with the dual-tree approach, but was developed for analysing ultrafast electron diffraction patterns. 22,28 The signal in these studies was generated by a mechanism orthogonal to those of spectroscopy, but given the favorable constellation of features and the formal equivalency of (powder) diffraction pattern and optical spectrum as consisting of peaks arrayed along an abscissa, [29][30][31] we undertook to explore the viability of the DTCWT-IA background subtraction applied to a range of spectroscopies: ultraviolet-visible (UV-Vis), SERS, and XPS. We performed background subtractions for these spectral types and compared the results to commonly used background subtractions often available as part of commercial analysis packages.…”

“…Specifically, DTCWT eliminates wavelet coefficient instability near sharp signal features, and aliasing artefacts that can arise from the use of real-valued wavelets. 22,28 It provides shift invariance and good directional selectivity. 15,23 Complex-valued wavelets are implemented by having one DWT tree, each, for the real and imaginary components, yielding the dual-tree approach.…”

“…15,23 Complex-valued wavelets are implemented by having one DWT tree, each, for the real and imaginary components, yielding the dual-tree approach. 22,28 Wavelet decomposition is a recursive process in which a spectrum is processed into ''smooth'' and ''detailed'' components, with further processing applied to the smooth components remaining after each recursion step. 2 The decomposition level of the method necessary to recover the (physical) background is determined by the frequency composition of the background.…”