All-day thin-lens computational imaging with scene-specific learning recovery

Abstract

Modern imaging optics ensures high-quality photography at the cost of a complex optical form factor that deviates from the portability. The drastic development of image processing algorithms, especially advanced neural networks, shows great promise to use thin optics but still faces the challenges of residual artifacts and chromatic aberration. In this work, we investigate photorealistic thin-lens imaging that paves the way to actual applications by exploring several fine-tunes. Notably, to meet all-day photography demands, we develop a scene-specific generative-adversarial-network-based learning strategy and develop an integral automatic acquisition and processing pipeline. Color fringe artifacts are reduced by implementing a chromatic aberration pre-correction trick. Our method outperforms existing thin-lens imaging work with better visual perception and excels in both normal-light and low-light scenarios.

Research advances in simple and compact optical imaging techniques

“…间，因此在简单透镜系统的优化过程中，也慢慢开始将衍射光学元件结合进来， 实现更高成像性能。Peng 等人提出一种折衍射混合式光学系统进行宽谱成像，并 通过反卷积算法实现球差和色差校正，为超薄镜片提供了替代技术解决方案，如 图 3 所示 [35] 。Wu 等人在相机光圈处添加相位掩膜实现单帧、单视点、被动三维 成像，同时采用一个端到端优化框架来联合优化相位掩膜和重构算法，并利用新 设计的相位掩膜搭建原理样机，在三维成像方面表现优异 [36] 。Chang 等人采用端 到端优化光学系统和图像处理的方法进行单目深度估计，使用编码的离焦模糊图 作为额外的深度线索，由神经网络解码，并评估了几种光学编码策略，验证了单 透镜的色差也可以显著提高深度估计的性能 [37] 。与分离式折衍射混合光学系统不 同，Peng 等人将衍射光学和折射光学结合，提出了一种菲涅耳透镜设计和学习重 建构架，仅使用单个薄板透镜元件就实现了 53°大视场成像 [38] 。此后，该团队又 设计了一款焦距更短、孔径更小的透镜，以匹配中小型传感器，通过色差预校正 9 技术减少彩色条纹伪影，实现了更好的视觉感知 [39] 。衍射光学元件具有高衍射效 率、色散性好、设计自由度大、成像功能丰富等优点，它为简单透镜成像系统带 来了更多可能，可促进光学系统的轻型化和智能化。 图3 折衍射混合式成像系统 [35] Fig. 3.…”

Review on Panoramic Imaging and Its Applications in Scene Understanding

“…Different from traditional optical imaging, computational imaging technology [211], [212] can encode and decode optical information captured by optical instruments in all directions from information acquisition, information transmission, and information conversion in terms of imaging principles. Computational imaging can acquire and analyze multi-dimensional information of the light field through scattering, polarization, and bionic technologies, and has many advantages in achieving large detection distance, high resolution [213], [214], high signal-to-noise ratio, multi-dimensional information [215], light weight [216], simplicity, and cheapness. For decades, optics researchers have been working to design compact optical systems with large FoV and light weight [217], [218].…”

Temperature‐Robust Learned Image Recovery for Shallow‐Designed Imaging Systems

“…As shown in Figure 3, for acquiring substantial datasets automatically, we built a display-capture setup [28,29] including a high-resolution, calibrated LCD monitor (Asus PA32U) and a shallow-designed phone module (Samsung GN1). Apart from the benefit of acquiring massive datasets without manual effort, it also considers the real noise and blur in both optical design and image signal processing (ISP) together instead of only dealing with one part of them.…”

“…Lightweight Architecture: We note that the depth of the recovery network depends on the degradation level, and images captured by the vehicle lens do not require such a deep network architecture, [29] termed as the Original model. Therefore, we can reduce the number of network blocks in the generator of the Original and construct a lightweight architecture as Current model.…”