Multifocusing and Depth Estimation Using a Color Shift Model-Based Computational Camera

Abstract

This paper presents a novel approach to depth estimation using a multiple color-filter aperture (MCA) camera and its application to multifocusing. An image acquired by the MCA camera contains spatially varying misalignment among RGB color channels, where the direction and length of the misalignment is a function of the distance of an object from the plane of focus. Therefore, if the misalignment is estimated from the MCA output image, multifocusing and depth estimation become possible using a set of image processing algorithms. We first segment the image into multiple clusters having approximately uniform misalignment using a color-based region classification method, and then find a rectangular region that encloses each cluster. For each of the rectangular regions in the RGB color channels, color shifting vectors are estimated using a phase correlation method. After the set of three clusters are aligned in the opposite direction of the estimated color shifting vectors, the aligned clusters are fused to produce an approximately in-focus image. Because of the finite size of the color-filter apertures, the fused image still contains a certain amount of spatially varying out-of-focus blur, which is removed by using a truncated constrained least-squares filter followed by a spatially adaptive artifacts removing filter. Experimental results show that the MCA-based multifocusing method significantly enhances the visual quality of an image containing multiple objects of different distances, and can be fully or partially incorporated into multifocusing or extended depth of field systems. The MCA camera also realizes single camera-based depth estimation, where the displacement between multiple apertures plays a role of the baseline of a stereo vision system. Experimental results show that the estimated depth is accurate enough to perform a variety of vision-based tasks, such as image understanding, description, and robot vision.

Reconstruction of multispectral light field (5d plenoptic function) based on compressive sensing with colored coded apertures from 2D projections

“…Algorithm 3 shows the process to train a dictionary ( Figure 6). [15], deblurring and matting [16], and compressive spectral imaging [17].…”

Superresolution Restoration From Directional Rectangular Blurred Images

“…최근에는 코드화된 조리개를 사용하여 해상도를 개선하 고 깊이 영상을 추정하는 계산사진학에 대한 연구도 활발 히 진행되고 있다 [6,7,8,9,10] . Mohan은 조리개 마스크의 위치 를 조정하며 촬영한 여러 장의 영상에서 고해상도 영상을 추정했다 [6] .…”

Multiple moving object segmentation using motion orientation histogram in adaptively partitioned blocks for high-resolution video surveillance systems

“…Segmentation and tracking of moving objects is a fundamental problem in high-level computer vision in such important applications as object-based auto-focusing, robot vision, human computer interfaces, activity recognition, and intelligent surveillance systems [1][2][3][4][5][6][7][8][9].…”