Deformable block-based motion estimation in omnidirectional image sequences

Simone, F.; Frossard, Pascal; Birkbeck, Neil; Adsumilli, Balu

doi:10.1109/mmsp.2017.8122254

Cited by 22 publications

(17 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover to the RGB or RGB-D image space, there also have been some researches about motion estimation for unique image space such as fisheye video sequence [16], and omnidirectional image space [17]. Eichenseer et al [16] proposed a hybrid block-based motion estimation method for real-world fisheye videos by combining perspective projection with considerations about the fisheye image structure.…”

Section: Related Workmentioning

confidence: 99%

“…Eichenseer et al [16] proposed a hybrid block-based motion estimation method for real-world fisheye videos by combining perspective projection with considerations about the fisheye image structure. Simone et al [17] presented an extension of block-based motion estimation for panoramic omnidirectional image sequences by considering spherical geometry of the imaging system. These researches focus on unique domains such as fisheye video and panoramic omnidirectional image sequences, so they are not applicable to the compound image domain which we are target for.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, to the best of our knowledge, we tackle the problem of ego-motion classification for compound eye cameras using convolutional neural network (CNN) for the first time, of which the goal is to classify the motion of compound eye camera given two consecutive emulated high resolution compound images. The ego-motion classification is the problem that focuses on the movement of the camera itself, which is different from the previous camera motion estimating algorithms that aim to estimate the transition between two scenes [12,13,14,15,16,17]. Here, the ego-motion classification of the compound eye camera gives an important contribution to the robot community, since knowing the moving direction of a robot is critical for problems such as localization and navigation.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep Ego-Motion Classifiers for Compound Eye Cameras

Yoo

Cha

2019

Sensors

View full text Add to dashboard Cite

Compound eyes, also known as insect eyes, have a unique structure. They have a hemispheric surface, and a lot of single eyes are deployed regularly on the surface. Thanks to this unique form, using the compound images has several advantages, such as a large field of view (FOV) with low aberrations. We can exploit these benefits in high-level vision applications, such as object recognition, or semantic segmentation for a moving robot, by emulating the compound images that describe the captured scenes from compound eye cameras. In this paper, to the best of our knowledge, we propose the first convolutional neural network (CNN)-based ego-motion classification algorithm designed for the compound eye structure. To achieve this, we introduce a voting-based approach that fully utilizes one of the unique features of compound images, specifically, the compound images consist of a lot of single eye images. The proposed method classifies a number of local motions by CNN, and these local classifications which represent the motions of each single eye image, are aggregated to the final classification by a voting procedure. For the experiments, we collected a new dataset for compound eye camera ego-motion classification which contains scenes of the inside and outside of a certain building. The samples of the proposed dataset consist of two consequent emulated compound images and the corresponding ego-motion class. The experimental results show that the proposed method has achieved the classification accuracy of 85.0%, which is superior compared to the baselines on the proposed dataset. Also, the proposed model is light-weight compared to the conventional CNN-based image recognition algorithms such as AlexNet, ResNet50, and MobileNetV2.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Ego-Motion Classifiers for Compound Eye Cameras

Yoo

Cha

2019

Sensors

View full text Add to dashboard Cite

show abstract

“…Motion estimation/compensation have already been studied for omnidirectional videos. First, they were developed for projected 360 • data (either with equirectangular [10,11,12,13], cubemap [14,15] or others [16,17]). Second, they propose different dimensionality reduction methods of the motion characterization, leading to various pros and cons.…”

Section: Introductionmentioning

confidence: 99%

Rate-Distortion Optimized Motion Estimation for on-the-Sphere Compression of 360 Videos

Marie

Bidgoli

Maugey

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

On-the-sphere compression of omnidirectional videos is a very promising approach. First, it saves computational complexity as it avoids to project the sphere onto a 2D map, as classically done. Second, and more importantly, it allows to achieve a better rate-distortion tradeoff, since neither the visual data nor its domain of definition are distorted. In this paper, the on-the-sphere compression [1] for omnidirectional still images is extended to videos. We first propose a complete review of existing spherical motion models. Then we propose a new one called tangent-linear+t. We finally propose a rate-distortion optimized algorithm to locally choose the best motion model for efficient motion estimation/compensation. For that purpose, we additionally propose a finer search pattern, called spherical-uniform, for the motion parameters, which leads to a more accurate block prediction. The novel algorithm leads to rate-distortion gains compared to methods based on a unique motion model.

show abstract

“…ARED 9582 InterCOR). motion compensation predictions are proposed in [6]- [8] to utilize sphere geometry in block matching technique. Another example is the transform proposed in [9], which is defined for equirectangular projected content.…”

Section: Introductionmentioning

confidence: 99%

Intra-coding of 360-degree images on the sphere

Bidgoli

Maugey

Roumy

2019

2019 Picture Coding Symposium (PCS)

View full text Add to dashboard Cite

Omni-directional images are characterized by their high resolution (usually 8K) and therefore require high compression efficiency. Existing methods project the spherical content onto one or multiple planes and process the mapped content with classical 2D video coding algorithms. However, this projection induces sub-optimality. Indeed, after projection, the statistical properties of the pixels are modified, the connectivity between neighboring pixels on the sphere might be lost, and finally, the sampling is not uniform. Therefore, we propose to process uniformly distributed pixels directly on the sphere to achieve high compression efficiency. In particular, a scanning order and a prediction scheme are proposed to exploit, directly on the sphere, the statistical dependencies between the pixels. A Graph Fourier Transform is also applied to exploit local dependencies while taking into account the 3D geometry. Experimental results demonstrate that the proposed method provides up to 5.6% bitrate reduction and on average around 2% bitrate reduction over state-of-the-art methods.

show abstract

Deformable block-based motion estimation in omnidirectional image sequences

Cited by 22 publications

References 11 publications

Deep Ego-Motion Classifiers for Compound Eye Cameras

Deep Ego-Motion Classifiers for Compound Eye Cameras

Rate-Distortion Optimized Motion Estimation for on-the-Sphere Compression of 360 Videos

Intra-coding of 360-degree images on the sphere

Contact Info

Product

Resources

About