Omnisupervised Omnidirectional Semantic Segmentation

Yang, Kailun; Hu, Xinxin; Fang, Yicheng; Wang, Kaiwei; Stiefelhagen, Rainer

doi:10.1109/tits.2020.3023331

Cited by 49 publications

(31 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Driving and aerial scene segmentation clearly benefit from expanded FoV, and standard semantic segmentation on pinhole images has been extended to operate on fisheye images [24], omnidirectional images [25], and panoramic annular images [26]. The latest progress include omni-supervised learning [27] and omni-range context modeling [28] to promote efficient 360 • driving scene understanding. In particular, Yang et al [26] proposed a generic framework by using a panoramic annular lens system, which intertwines a network adaptation method by re-using models learned on standard semantic segmentation datasets.…”

Section: B Panoramic Scene Segmentationmentioning

confidence: 99%

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

Sun

Wang

Yang

et al. 2021

2021 European Conference on Mobile Robots (ECMR)

Self Cite

View full text Add to dashboard Cite

Aerial pixel-wise scene perception of the surrounding environment is an important task for UAVs (Unmanned Aerial Vehicles). Previous research works mainly adopt conventional pinhole cameras or fisheye cameras as the imaging device. However, these imaging systems cannot achieve large Field of View (FoV), small size, and lightweight at the same time. To this end, we design a UAV system with a Panoramic Annular Lens (PAL), which has the characteristics of small size, low weight, and a 360 • annular FoV. A lightweight panoramic annular semantic segmentation neural network model is designed to achieve high-accuracy and real-time scene parsing. In addition, we present the first drone-perspective panoramic scene segmentation dataset Aerial-PASS, with annotated labels of track, field, and others. A comprehensive variety of experiments shows that the designed system performs satisfactorily in aerial panoramic scene parsing. In particular, our proposed model strikes an excellent trade-off between segmentation performance and inference speed suitable, validated on both public street-scene and our established aerial-scene datasets.

show abstract

Section: B Panoramic Scene Segmentationmentioning

confidence: 99%

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

Sun

Wang

Yang

et al. 2021

2021 European Conference on Mobile Robots (ECMR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Yang et al introduce the PASS [7] and the DS-PASS [44] frameworks which naturally mitigate the effect of distortions by using a single-shot panoramic annular lens system, but come with an expensive memory-and computation cost, as it requires separating the panorama into multiple partitions for predictions, each resembling a narrow-FoV pinhole image. This is significantly improved by the OOSS framework [45] through multi-source omni-supervised learning. The latest advancements include frameworks focusing on dimension-wise positional priors [46], omni-range contextual dependencies [33], or leveraging contrastive pixel-propagation pre-training [47].…”

Section: B Semantic Segmentation For 360 • Panoramic Imagesmentioning

confidence: 99%

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation

Zhang¹,

Ma²,

Yang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

“…Roddick et al [28] and Philion et al [29] incorporated the strong geometric priors of camera extrinsic parameters into the pipeline, which presents impressive performance. Moreover, Deng et al [30] and Yang et al [31] used fisheye images to generate semantic predictions in BEV space. The semantic map forecasting module in our approach is inspired by these works.…”

Section: B Semantic Segmentation In Bev Spacementioning

confidence: 99%

Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation

Wang

Cai

Sun

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, deep-learning based approaches have achieved impressive performance for autonomous driving. However, end-to-end vision-based methods typically have limited interpretability, making the behaviors of the deep networks difficult to explain. Hence, their potential applications could be limited in practice. To address this problem, we propose an interpretable end-to-end vision-based motion planning approach for autonomous driving, referred to as IVMP. Given a set of past surrounding-view images, our IVMP first predicts future egocentric semantic maps in bird's-eye-view space, which are then employed to plan trajectories for self-driving vehicles. The predicted future semantic maps not only provide useful interpretable information, but also allow our motion planning module to handle objects with low probability, thus improving the safety of autonomous driving. Moreover, we also develop an optical flow distillation paradigm, which can effectively enhance the network while still maintaining its real-time performance. Extensive experiments on the nuScenes dataset and closed-loop simulation show that our IVMP significantly outperforms the state-of-the-art approaches in imitating human drivers with a much higher success rate. Our project page is available at https://sites.google.com/view/ivmp.

show abstract

Omnisupervised Omnidirectional Semantic Segmentation

Cited by 49 publications

References 60 publications

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation

Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation

Contact Info

Product

Resources

About