Multi-View Representation Learning with Manifold Smoothness

Abstract

Multi-view representation learning attempts to learn a representation from multiple views and most existing methods are unsupervised. However, representation learned only from unlabeled data may not be discriminative enough for further applications (e.g., clustering and classification). For this reason, semi-supervised methods which could use unlabeled data along with the labeled data for multi-view representation learning need to be developed. Manifold information plays an important role in semi-supervised learning, but it has not been considered for multi-view representation learning. In this paper, we introduce the manifold smoothness into multi-view representation learning and propose MvDGAT which learns the representation and the intrinsic manifold simultaneously with graph attention network. Experiments conducted on real-world datasets reveal that our MvDGAT can achieve better performance than state-of-the-art methods.

Multi-Camera Collaborative Depth Prediction via Consistent Structure Estimation

“…Depth estimation plays a crucial role in 3D perception, significantly influencing the performance of downstream tasks within the perception system. Numerous works in visionbased detection [53], [54], semantic occupancy prediction [51], [55], and bird's-eye-view representations [56] integrate a depth estimation network as a foundational component. These studies collectively affirm that the precision of depth estimation directly correlates with the performance of subsequent tasks.…”

“…BevDepth [53] illustrates the profound impact of employing an explicit depth estimation strategy on 3D object detection accuracy. BevDepth's a dedicated branch network undertakes depth estimation by back-projecting 2D object features into 3D space.…”

“…Specificly, following the operation in BevDepth [53], we employ a pretrained VoVNet [57] as the backbone on the nuScenes dataset [11], and construct a 3D object detection baseline. The depth estimation branch calculates the crossentropy loss between the predicted depth and ground-truth depth at a resolution of 640 × 1600.…”

“…OF 3D DETECTION RESULTS BETWEEN BASELINE AND ADDING DEPTH PRIOR ON THE NUSCENES TEST SET[11]. TAKING BEVDEPTH[53] AS THE BASELINE, WE ADD DEPTH PRIOR AS INPUT AND SUPERVISION IN THE DEPTH ESTIMATION BRANCH, AND VERIFY THE EFFECTIVENESS OF DEPTH ESTIMATION THROUGH THE RESULTS OF 3D OBJECT DETECTION.TABLE X RESULTS OF 3D OCCUPANCY PREDICTION ON THE OCC3D-NUSCENES [52] DATASET. WE USE VOXFORMER [51] AS BASELINE AND APPLY OUR PROPOSED METHOD IN DEPTH ESTIMATION BRANCH NETWORK.…”