Enhancement of sparse silicon retina-based stereo matching using belief propagation and two-stage postfiltering

Kogler, Jürgen; Eibensteiner, Florian; Humenberger, Martin; Sulzbachner, Christoph; Gelautz, Margrit; Scharinger, Josef

doi:10.1117/1.jei.23.4.043011

Cited by 17 publications

(10 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A discussion of cooperative stereo is provided in [43]. Also in this category are [186], [187], [188], which use Belief Propagation on a Markov Random Field or semiglobal matching [189] to improve stereo matching. These methods are primarily based on optimization, trying to define a well-behaved energy function whose minimizer is the correct correspondence map.…”

Section: D Reconstruction Monocular and Stereomentioning

confidence: 99%

Event-Based Vision: A Survey

Gallego

Delbrück

Orchard

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

1,298

720

View full text Add to dashboard Cite

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of µs), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.

show abstract

Section: D Reconstruction Monocular and Stereomentioning

confidence: 99%

Event-Based Vision: A Survey

Gallego

Delbrück

Orchard

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

1,298

720

View full text Add to dashboard Cite

show abstract

“…Kogler, who tried for a long time to apply classical algorithms to event-based data, offers in Kogler et al (2014) an alternative realization of event-based stereo vision with Belief Propagation. He complements this with a subsequent filtering in two phases.…”

Section: Event-driven Stereoscopymentioning

confidence: 99%

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

et al. 2019

View full text Add to dashboard Cite

Any visual sensor, whether artificial or biological, maps the 3D-world on a 2D-representation. The missing dimension is depth and most species use stereo vision to recover it. Stereo vision implies multiple perspectives and matching, hence it obtains depth from a pair of images. Algorithms for stereo vision are also used prosperously in robotics. Although, biological systems seem to compute disparities effortless, artificial methods suffer from high energy demands and latency. The crucial part is the correspondence problem; finding the matching points of two images. The development of event-based cameras, inspired by the retina, enables the exploitation of an additional physical constraint—time. Due to their asynchronous course of operation, considering the precise occurrence of spikes, Spiking Neural Networks take advantage of this constraint. In this work, we investigate sensors and algorithms for event-based stereo vision leading to more biologically plausible robots. Hereby, we focus mainly on binocular stereo vision.

show abstract

“…According to the criterion for disparity estimation, they can be classified into local matching and global matching categories [3]. The global matching methods include dynamic programming (DP) [17]- [19], belief propagation [20], [21] and graph cut (GC) [22], [23]. The local methods include the region-based matching [24], phase-based matching [25] and feature-based matching [26] algorithms.…”

Section: Related Workmentioning

confidence: 99%

A Stereo Matching and Depth Map Acquisition Algorithm Based on Deep Learning and Improved Winner Takes All-Dynamic Programming

Zhu

Yan

2019

IEEE Access

View full text Add to dashboard Cite

The stereo matching technology has been widely used nowadays for robot guidance, depth of field rendering, and video processing applications. The binocular stereo matching technology collects two images of the same scene from different views, computes the disparity, and then determines the 3-D depth of the object using the triangulation theory. In this paper, we propose a new deep-learning-based stereo matching algorithm called convolutional neural network WTADP (CNN-WTADP). The commonly used WTA method ignores the disparity constraint of the neighboring pixels and is prone to be affected by noise, so the obtained original disparity map may contain a lot of abnormal values and mismatched points. To solve this problem, we construct a CNN for matching cost computation and use a novel global method of dynamic programming based on WTA for disparity selection. Steps for the CNN-based computation of the matching cost are presented, and the post-processing algorithm is used to obtain the final determination of the disparity map. The Middlebury stereo dataset and stereo evaluation are used to compare the proposed algorithm with other methods. The experiments and its results demonstrate the ability of the proposed algorithm to match the images more effectively. INDEX TERMS Deep learning, neural network, stereo matching.

show abstract

Enhancement of sparse silicon retina-based stereo matching using belief propagation and two-stage postfiltering

Cited by 17 publications

References 48 publications

Event-Based Vision: A Survey

Event-Based Vision: A Survey

Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms

A Stereo Matching and Depth Map Acquisition Algorithm Based on Deep Learning and Improved Winner Takes All-Dynamic Programming

Contact Info

Product

Resources

About