NoVA: Learning to See in Novel Viewpoints and Domains

Coors, Benjamin; Condurache, Alexandru Paul; Geiger, Andreas

doi:10.1109/3dv.2019.00022

Cited by 16 publications

(9 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The domain adaptation problem has received great interest in recent years, especially in the unsupervised setting, with several studies in the context of diverse tasks such as classification [7], [4], semantic segmentation [8], [9], [10] and more recently object detection [11], [12], [13]. In the following, we review the most related work on unsupervised domain adaptation in general and in the context of object detection specifically, as well as cross-domain learning with weak supervision.…”

Section: Related Workmentioning

confidence: 99%

Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation

Hanselmann¹,

Schneider²,

Ortelt³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In order to handle the challenges of autonomous driving, deep learning has proven to be crucial in tackling increasingly complex tasks, such as 3D detection or instance segmentation. State-of-the-art approaches for image-based detection tasks tackle this complexity by operating in a cascaded fashion: they first extract a 2D bounding box based on which additional attributes, e.g. instance masks, are inferred. While these methods perform well, a key challenge remains the lack of accurate and cheap annotations for the growing variety of tasks. Synthetic data presents a promising solution but, despite the effort in domain adaptation research, the gap between synthetic and real data remains an open problem. In this work, we propose a weakly supervised domain adaptation setting which exploits the structure of cascaded detection tasks. In particular, we learn to infer the attributes solely from the source domain while leveraging 2D bounding boxes as weak labels in both domains to explain the domain shift. We further encourage domain-invariant features through class-wise feature alignment using ground-truth class information, which is not available in the unsupervised setting. As our experiments demonstrate, the approach is competitive with fully supervised settings while outperforming unsupervised adaptation approaches by a large margin.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation

Hanselmann¹,

Schneider²,

Ortelt³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Unlike cross-view image classification [36,63,10,1,16], aligning domains of different viewpoints for pixel-level prediction tasks is ill-posed, since the task is indeed view dependent [7]. The most relevant are [11,8], which again resort to adversarial domain alignment. Additionally, [8] requires known camera intrinsics and extrinsics.…”

Section: Related Workmentioning

confidence: 99%

“…The most relevant are [11,8], which again resort to adversarial domain alignment. Additionally, [8] requires known camera intrinsics and extrinsics. Note, both assume the viewpoint change is small or there is a large overlap between the two views, therefore the applicability to a broader setting is limited, whereas our method is not constrained by any of these assumptions.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

Yang,

Ren,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

“…Crucially, a BEV enables to efficiently transform observations between varying points of view and estimate agent-centric states when learning by watching. In contrast, transforming other types of representations across views, e.g., [16,46,77], can be difficult for significantly differing perspectives.…”

Section: Intermediate Representations For Autonomous Drivingmentioning

confidence: 99%

Learning by Watching

Zhang¹,

Ohn-Bar²

2021

Preprint

View full text Add to dashboard Cite

When in a new situation or geographical location, human drivers have an extraordinary ability to watch others and learn maneuvers that they themselves may have never performed. In contrast, existing techniques for learning to drive preclude such a possibility as they assume direct access to an instrumented ego-vehicle with fully known observations and expert driver actions. However, such measurements cannot be directly accessed for the non-ego vehicles when learning by watching others. Therefore, in an application where data is regarded as a highly valuable asset, current approaches completely discard the vast portion of the training data that can be potentially obtained through indirect observation of surrounding vehicles. Motivated by this key insight, we propose the Learning by Watching (LbW) framework which enables learning a driving policy without requiring full knowledge of neither the state nor expert actions. To increase its data, i.e., with new perspectives and maneuvers, LbW makes use of the demonstrations of other vehicles in a given scene by (1) transforming the egovehicle's observations to their points of view, and (2) inferring their expert actions. Our LbW agent learns more robust driving policies while enabling data-efficient learning, including quick adaptation of the policy to rare and novel scenarios. In particular, LbW drives robustly even with a fraction of available driving data required by existing methods, achieving an average success rate of 92% on the original CARLA benchmark with only 30 minutes of total driving data and 82% with only 10 minutes.

show abstract

NoVA: Learning to See in Novel Viewpoints and Domains

Cited by 16 publications

References 26 publications

Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation

Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

Learning by Watching

Contact Info

Product

Resources

About