TEM Video Compressive Sensing

Abstract

One of the main limitations of imaging at high spatial and temporal resolution during in-situ TEM experiments is the frame rate of the camera being used to image the dynamic process. While the recent development of direct detectors has provided the hardware to achieve frame rates approaching 0.1ms, the cameras are expensive and must replace existing detectors. In this paper, we examine the use of coded aperture compressive sensing methods [1,2,3,4] to increase the framerate of any camera with simple, low-cost hardware modifications. The coded aperture approach allows multiple sub-frames to be coded and integrated into a single camera frame during the acquisition process, and then extracted upon readout using statistical compressive sensing inversion. Our simulations show that it should be possible to increase the speed of any camera by at least an order of magnitude.Compressive Sensing (CS) combines sensing and compression in one operation, and thus provides an approach that could further improve the temporal resolution while correspondingly reducing the electron dose rate. Because the signal is measured in a compressive manner, fewer total measurements are required. When applied to TEM video capture, compressive imaging could improve acquisition speed and reduce the electron dose rate. CS is a recent concept, and has come to the forefront due the seminal work of Candès [5]. Since the publication of Candès, there has been enormous growth in the application of CS and development of CS variants. For electron microscopy applications, the concept of CS has also been recently applied to electron tomography [6], and reduction of electron dose in scanning transmission electron microscopy (STEM) imaging [7].To demonstrate the applicability of coded aperture CS video reconstruction for atomic level imaging, we simulate compressive sensing on observations of Pd nanoparticles and Ag nanoparticles during exposure to high temperatures and other environmental conditions. Figure 1 highlights the results from the Pd nanoparticle experiment. On the left, 10 frames are reconstructed from a single coded frame-the original frames are shown for comparison. On the right a selection of three frames are shown from reconstructions at compression levels 10, 20, 30. The reconstructions, which are not post-processed, are true to the original and degrade in a straightforward manner. The final choice of compression level will obviously depend on both the temporal and spatial resolution required for a specific imaging task, but the results indicate that an increase in speed of better than an order of magnitude should be possible for all experiments [8].References:[1] P Llull, X Liao, X Yuan et al.

Compressive video sensing with side information

“…Inspired by theoretical developments in compressive sensing [1], several compressive imaging systems in the time domain have been conceived and implemented, aiming to extract high-frame-rate videos from low-speed image sequences [2][3][4][5][6][7][8][9][10][11][12][13]. In these systems, to recover the high-frame-rate video, the measurements are encoded by a flutter shutter [2,3], a physical mask [4][5][6], or a digital mirror array [7][8][9][10] whose pattern varies at a rate faster than the frame rate of the camera.…”

“…In these systems, to recover the high-frame-rate video, the measurements are encoded by a flutter shutter [2,3], a physical mask [4][5][6], or a digital mirror array [7][8][9][10] whose pattern varies at a rate faster than the frame rate of the camera. This technique has been extended to microscopy imaging [11][12][13]. Video compressive sensing based on the spatialmultiplexing camera has also been investigated [14,15].…”

Implementing Sparse Sub-Sampling Methods for Low-Dose/High Speed STEM

“…Technological limitations are also associated with obtaining images faster; in the case of the TEM lowdose imaging requirements exceed the framerate of the camera, while in STEM the hysteresis in the scan coils limits the overall scanning speed that can be obtained. To overcome these limitations, image reconstruction methods such as compressive sensing and in-painting [1,2] can be used to reduce the number of read-outs/pixels in the STEM images/movies and lower the overall electron dose while at the same time increasing the speed of the image and reducing the overall size of the dataset acquired [3,4].…”

Compressive Sensing in Microscopy: a Tutorial

“…An alternative solution, which additionally benefits from improved hardware, is to apply compressive sensing (CS) [1]. CS approaches have been shown to reduce dose by as much as 90% in electron microscopy [2,3,4]. Optical imaging and microscopy have also seen substantial benefits [5,6,7,8,9,10,11,12,13].…”