Advanced Three-Dimensional Visualization System for an Integral Imaging Microscope Using a Fully Convolutional Depth Estimation Network

Abstract

In this paper, we propose an advanced three-dimensional visualization method for an integral imaging microscope system to simultaneously improve the resolution and quality of the reconstructed image. The main advance of the proposed method is that it generates a high-quality three-dimensional model without limitation of resolution by combining the high-resolution two-dimensional color image with depth data obtained through a fully convolutional neural network. First, the high-resolution twodimensional image and an elemental image array for a specimen are captured, and the orthographic-view image is reconstructed from the elemental image array. Then, via a convolutional neural network-based depth estimation after the brightness of input images are uniformed, a more accurate and improved depth image is generated; and the noise of result depth image is filtered. Subsequently, the estimated depth data is combined with the high-resolution two-dimensional image and transformed into a highquality three-dimensional model. In the experiment, it was confirmed that the displayed high-quality three-dimensional model could be visualized very similarly to the original image.

Depth Estimation for Integral Imaging Microscopy Using a 3D–2D CNN with a Weighted Median Filter

“…Using surface subdivision-based points interpolation, a 3D point cloud of the gear, microchip, and seedpod was created using 2D images and the proposed depth map. In addition to the gear, the microchip, and the seedpod were regenerated using 3D point clouds (562,500 object points for the gear, 577,600 object points for the microchip, and 291,600 object points for the seedpod) and a surface subdivision-based interpolation method [42], which included nearly 10-20-fold more points than the original generation. images and the proposed depth map.…”

“…images and the proposed depth map. In addition to the gear, the microchip, and the seedpod were regenerated using 3D point clouds (562,500 object points for the gear, 577,600 object points for the microchip, and 291,600 object points for the seedpod) and a surface subdivision-based interpolation method [42], which included nearly 10-20-fold more points than the original generation. Table 2 presents the PSD values for 2D images, central image of directional-view images, and central views for the reconstructed object models in the point cloud of different methods.…”

“…The entire procedure is separated into three parts: image capture through IIM system with camera calibration [ 41 ], depth estimation using 3D convolutional neural network, and filtering by weighted median to eliminate depth noise [ 42 ]. During the capture process, the specimen is positioned in front of the microscope objective lens.…”

“…The training loss and validation graph is shown in Figure 6 . Tests are performed using the real light-field IIMs image provided by Kwon et al [ 42 ]. The experiment was conducted with three types of real object reconstruction of the OVIs, as shown in Figure 6 .…”

Classification Method for Power Load Data of New Energy Grid based on Improved OTSU Algorithm

“…The power load data of new energy grid can be visually displayed through the current monitoring data, and the current can be regarded as a one-dimensional sequence related to time in essence. According to the idea of two-dimensional visualization of time series [12], GAF method is introduced into the power consumption field of new energy grid to build a twodimensional visualization image for the power load data of new energy grid. Assuming that   12 , , , n X…”