Joint object recognition and pose estimation using a nonlinear view-invariant latent generative model

Bakry, Amr; El-Gaaly, Tarek; Elhoseiny, Mohamed; Elgammal, Ahmed

doi:10.1109/wacv.2016.7477655

Cited by 6 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two main lines of research that are relevant to our work: (i) 3D pose estimation given object category label and (ii) joint object category and pose estimation. There are many non-deep learning based approaches such as [12,13,14,15,16,17,18,19], which have designed systems to solve these two tasks. However, due to space constraints, we restrict our discussion to deep learning based approaches.…”

Section: Related Workmentioning

confidence: 99%

Convolutional Networks for Object Category and 3D Pose Estimation from 2D Images

Mahendran

Ali

Vidal

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Current CNN-based algorithms for recovering the 3D pose of an object in an image assume knowledge about both the object category and its 2D localization in the image. In this paper, we relax one of these constraints and propose to solve the task of joint object category and 3D pose estimation from an image assuming known 2D localization. We design a new architecture for this task composed of a feature network that is shared between subtasks, an object categorization network built on top of the feature network, and a collection of category dependent pose regression networks. We also introduce suitable loss functions and a training method for the new architecture. Experiments on the challenging PASCAL3D+ dataset show state-of-the-art performance in the joint categorization and pose estimation task. Moreover, our performance on the joint task is comparable to the performance of state-of-the-art methods on the simpler 3D pose estimation with known object category task.

show abstract

Section: Related Workmentioning

confidence: 99%

Convolutional Networks for Object Category and 3D Pose Estimation from 2D Images

Mahendran

Ali

Vidal

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The shared first five layers are able to build up the object-view manifolds, preserve them and enhance them in the pose subnetwork of the model, while the other subnetwork specializes in pose-invariant category recognition. Approach Category % Pose (AAAI) % (Lai et al, 2011b) 94.30 (RGB + Depth) 53.50 (Bakry & Elgammal, 2014) 94.84 (RGB only)/ 96.01 (RGB+ Depth) 76.01 (Zhang et al, 2013a) 92.00 (RGB only)/ 93.10 (RGB + Depth) 61.57 (Bakry et al, 2016) 85.00 77.31 Ours (EBM(800))…”

Section: Computational Analysis and Convergencementioning

confidence: 99%

Convolutional Models for Joint Object Categorization and Pose Estimation

Elhoseiny¹,

El-Gaaly²,

Bakry³

et al. 2015

Preprint

Self Cite

View full text Add to dashboard Cite

In the task of Object Recognition, there exists a dichotomy between the categorization of objects and estimating object pose, where the former necessitates a view-invariant representation, while the latter requires a representation capable of capturing pose information over different categories of objects. With the rise of deep architectures, the prime focus has been on object category recognition. Deep learning methods have achieved wide success in this task. In contrast, object pose regression using these approaches has received relatively much less attention. In this paper we show how deep architectures, specifically Convolutional Neural Networks (CNN), can be adapted to the task of simultaneous categorization and pose estimation of objects. We investigate and analyze the layers of various CNN models and extensively compare between them with the goal of discovering how the layers of distributed representations of CNNs represent object pose information and how this contradicts object category representations. We extensively experiment on two recent large and challenging multi-view datasets. Our models achieve better than state-of-the-art performance on both datasets.

show abstract

“…In computer vision tasks, while deep learning models are known to be vulnerable to adversarial perturbations in pixel values (Szegedy et al 2014;Croce and Hein 2020;Yin, Ruan, and Fieldsend 2022;Mu et al 2021Mu et al , 2022, Engstrom et al (2019) show that a slight rotation of an input example can also fool DNNs. Although modern DNNs are believed to be able to learn geometric information from training data (Bakry et al 2016), they are not yet invariant to simple adversarial geometric transformations (Zhang et al 2020).…”

Section: Introductionmentioning

confidence: 99%

Towards Verifying the Geometric Robustness of Large-Scale Neural Networks

Wang

Ruan

et al. 2023

AAAI

View full text Add to dashboard Cite

Deep neural networks (DNNs) are known to be vulnerable to adversarial geometric transformation. This paper aims to verify the robustness of large-scale DNNs against the combination of multiple geometric transformations with a provable guarantee. Given a set of transformations (e.g., rotation, scaling, etc.), we develop GeoRobust, a black-box robustness analyser built upon a novel global optimisation strategy, for locating the worst-case combination of transformations that affect and even alter a network's output. GeoRobust can provide provable guarantees on finding the worst-case combination based on recent advances in Lipschitzian theory. Due to its black-box nature, GeoRobust can be deployed on large-scale DNNs regardless of their architectures, activation functions, and the number of neurons. In practice, GeoRobust can locate the worst-case geometric transformation with high precision for the ResNet50 model on ImageNet in a few seconds on average. We examined 18 ImageNet classifiers, including the ResNet family and vision transformers, and found a positive correlation between the geometric robustness of the networks and the parameter numbers. We also observe that increasing the depth of DNN is more beneficial than increasing its width in terms of improving its geometric robustness. Our tool GeoRobust is available at https://github.com/TrustAI/GeoRobust.

show abstract

Joint object recognition and pose estimation using a nonlinear view-invariant latent generative model

Cited by 6 publications

References 24 publications

Convolutional Networks for Object Category and 3D Pose Estimation from 2D Images

Convolutional Networks for Object Category and 3D Pose Estimation from 2D Images

Convolutional Models for Joint Object Categorization and Pose Estimation

Towards Verifying the Geometric Robustness of Large-Scale Neural Networks

Contact Info

Product

Resources

About