An accurate and fast reconstruction algorithm is crucial for the improvement of temporal resolution in high-density super-resolution microscopy, particularly in view of the challenges associated with live-cell imaging. In this work, we design a deep network based on a convolutional neural network to take advantage of its enhanced ability in high-density molecule localization, and introduce a residual layer into the network to reduce noise. The proposed scheme also incorporates robustness against variations of both the full width at half maximum (FWHM) and the pixel size. We validate our algorithm on both simulated and experimental data by achieving performance improvement in terms of loss value and image quality, and demonstrate live-cell imaging with temporal resolution of 0.5 seconds by recovering mitochondria dynamics.

Convolutional neural network (CNN) has been widely adopted in many tasks. Its inference process is usually applied on edge devices where the computing resources and power consumption are limited. At present, the performance of general processors cannot meet the requirement for CNN models with high computation complexity and large number of parameters. Field‐programmable gate array (FPGA)‐based custom computing architecture is a promising solution to further enhance the CNN inference performance. The software/hardware co‐design can effectively reduce the computing overhead, and improve the inference performance while ensuring accuracy. In this paper, the mainstream methods of CNN structure design, hardware‐oriented model compression and FPGA‐based custom architecture design are summarized, and the improvement of CNN inference performance is demonstrated through an example. Challenges and possible research directions in the future are concluded to foster research efforts in this domain.

In order to efficiently remove honeycomb artifacts and restore images in fiber-bundle-based endomicroscopy, we develop a meta-learning algorithm in this work. Two sub-networks are used to extract different levels of features. Meta-training is employed to train the network with small amount of simulated training data, enabling the optimal model to generalize to new tasks not seen in the training set. Numerical results on both USAF target and endomicroscopy images of living mice tissues demonstrate that the algorithm can restore high contrast image without pixilated noise using shorter time. Additionally, no prior information on the shape of the underlying tissues and the distribution of fiber bundles is required, making the method applicable to a variety of fiber-bundle-based endomicroscopy imaging conditions.

Super-resolution optical imaging is crucial to the study of cellular processes. Current super-resolution fluorescence microscopy is restricted by the need of special fluorophores or sophisticated optical systems, or long acquisition and computational times. In this work, we present a deep-learning-based super-resolution technique of confocal microscopy. We devise a two-channel attention network (TCAN), which takes advantage of both spatial representations and frequency contents to learn a more precise mapping from low-resolution images to high-resolution ones. This scheme is robust against changes in the pixel size and the imaging setup, enabling the optimal model to generalize to different fluorescence microscopy modalities unseen in the training set. Our algorithm is validated on diverse biological structures and dual-color confocal images of actin-microtubules, improving the resolution from ~ 230 nm to ~ 110 nm. Last but not least, we demonstrate live-cell super-resolution imaging by revealing the detailed structures and dynamic instability of microtubules.

No abstract