Multimodal image fusion with SIMS: Preprocessing with image registration

Abstract

In order to utilize complementary imaging techniques to supply higher resolution data for fusion with secondary ion mass spectrometry (SIMS) chemical images, there are a number of aspects that, if not given proper consideration, could produce results which are easy to misinterpret. One of the most critical aspects is that the two input images must be of the same exact analysis area. With the desire to explore new higher resolution data sources that exists outside of the mass spectrometer, this requirement becomes even more important. To ensure that two input images are of the same region, an implementation of the insight segmentation and registration toolkit (ITK) was developed to act as a preprocessing step before performing image fusion. This implementation of ITK allows for several degrees of movement between two input images to be accounted for, including translation, rotation, and scale transforms. First, the implementation was confirmed to accurately register two multimodal images by supplying a known transform. Once validated, two model systems, a copper mesh grid and a group of RAW 264.7 cells, were used to demonstrate the use of the ITK implementation to register a SIMS image with a microscopy image for the purpose of performing image fusion.

DCNN Optimization using Wavelet-based Image Fusion

“…Multi-Focus Fusion has applications in a wide array of fields. Aerial surveillance, three-dimensional reconstruction, photography, and video production all have ways to benefit from this fusion technique [19,20,21]. Multi-perspective fusion, shown in Figure 1, involves different photos taken of the same object or scene from different angles.…”

DCNN Optimization Using Multi-Resolution Image Fusion

“…After the images have been decomposed on multiple levels, these coefficients are fused through various methods such as maximum, minimum, mean, or principal component analysis. The wavelets applied in this study include Coiflet, biorthogonal, Meyer, Daubechies, Symlet, and Gaussian derivatives [14,15,16,17,18,19].…”

Application of pan-sharpening algorithm for correlative multimodal imaging using AFM-IR

“…This approach can be used in the case when the functional relationship between two images (e.g., how change of one parameter is related to the change of the second one) is not precisely know a priori and needs to be established during the analysis. However, data fusion of mass spectrometry data with microscopy data that has already been demonstrated [28][29][30] is prone to the generation of reconstruction artifacts. Specifically, the image formation mechanism in a secondary electron or optical image is drastically different from chemically-sensitive spectroscopical channels, so the correlation established within data fusion process implies a relationship between two channels that are not physically linked together.…”