Dynamic multifoveated structure for real-time vision tasks in robotic systems

Abstract

Foveation is a technique that allows real-time image processing by drastically reducing the amount of visual data without loosing essential information around some focused area. When a robot needs to pay attention at two or more regions of the image at the same time, e.g., for tracking two or more objects, multifoveation is necessary. In this case, computing features twice in the intersections between the different foveated structures, which could linearly increase the processing time, must be avoided. To solve this redundancy removal problem, we propose two algorithms. The first one is based on the previous calculation of redundant blocks and the second one is based on a pixel-by-pixel processing at execution time. Experimental results show a gain in processing time for the block-based model in comparison with the pixel-by-pixel and also of both in comparison with other approaches that sequentially calculate various single foveated images. Robotics vision and other tasks related to dynamic visual attention, as recognition, real-time surveillance, video transmission, and image rendering, are examples of applications that can rely on and strongly benefit from such model.

Targets Detection Using Multiple Foveas

“…Medeiros et al [21], [22] proposed the multifoveation by replication of the MMF structure, but that results in high computational costs since the structures are applied independently, leading to redundant processing of the image's portions. They solved this problem by removing the redundant processing between foveas (see Figure 3).…”