Heterogeneous Multi-View Information Fusion: Review of 3-D Reconstruction Methods and a New Registration with Uncertainty Modeling

Aliakbarpour, Hadi; Prasath, V. B. Surya; Palaniappan, Kannappan; Seetharaman, Guna; Dias, Jorge

doi:10.1109/access.2016.2629987

Cited by 25 publications

(19 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides positioning, a seamless handover mechanism is proposed by fusing GPS information of mobile terminals and received signals from the mobile terminals [26]. In addition to the fusion of RF and visual modalities, methods for fusing multiple streams of visual data are extensively studied in the literature to reconstruct three-dimensional objects and scenes [27] or to compensate for single camera's limited FoV [21], [28] or occlusions [29]. However, these studies do not consider the privacy in collecting highly private sensitive information, e.g., trajectory of humans viewed in visual data or mobile users tracked by GPS.…”

Section: B Related Work and Organizationmentioning

confidence: 99%

Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction

et al. 2020

View full text Add to dashboard Cite

The goal of this study is to improve the accuracy of millimeter wave received power prediction by utilizing camera images and radio frequency (RF) signals, while gathering image inputs in a communication-efficient and privacy-preserving manner. To this end, we propose a distributed multimodal machine learning (ML) framework, coined multimodal split learning (MultSL), in which a large neural network (NN) is split into two wirelessly connected segments. The upper segment combines images and received powers for future received power prediction, whereas the lower segment extracts features from camera images and compresses its output to reduce communication costs and privacy leakage. Experimental evaluation corroborates that MultSL achieves higher accuracy than the baselines utilizing either images or RF signals. Remarkably, without compromising accuracy, compressing the lower segment output by 16x yields 16x lower communication latency and 2.8% less privacy leakage compared to the case without compression.Index Terms-Millimeter-wave communications, received power prediction, multi-modal deep learning, split learning. Received powersReccurent neural network mmWave UE mmWave BS Future received power at BS Uplink dataGround-truth

show abstract

Section: B Related Work and Organizationmentioning

confidence: 99%

Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction

et al. 2020

View full text Add to dashboard Cite

show abstract

“…where the coefficients s c and s i are the scale coefficients with regard to camera error term and IMU error term. The camera feature re-projection error E C (t, q) between camera frame t and frame q is given by (15) where f t,j and f q,j are the j th feature point locations at image t and image q respectively; π (•) denotes the feature reprojection at image domain; q t,j is the information matrix corresponding to the j th feature correspondence. For further details in computing the re-projection errors and information matrix, please refer to [19].…”

Section: A Vehicle Dynamic Analysismentioning

confidence: 99%

“…Complementary to vehicle cameras, inertial measurement units (IMUs) are also able to provide meaningful information for assistive driving. As a proprioceptive sensor, IMU has the heterogeneous and complementary characteristics to cameras, which, by contrast, belong to exteroceptive categories [15]. In general, an IMU commonly consists of tri-axes accelerometers, tri-axes gyroscopes and tri-axes magnetometers.…”

Section: Introductionmentioning

confidence: 99%

Robust Vehicle and Surrounding Environment Dynamic Analysis for Assistive Driving Using Visual-Inertial Measurements

Zhang

Liang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Vehicle and surrounding environment dynamic analysis (VSEDA) is an indispensable component of modern assistive drivings. A robust and accurate VSEDA could ensure the driving system reliability in presence of highly dynamic environments. This paper proposes a novel VSEDA framework by fusing the measurements from an inertial sensor and a monocular camera. Compared to traditional visualinertial-based assistive driving methods, the proposed approach can analyze both the vehicle dynamics and the surrounding environment. Even in the scenario that moving objects occupy a majority area of the scene captured in the image, the proposed method can still robustly analyze the surrounding environment by identifying the static inliers and dynamic inliers, which lie on stationary objects and moving objects, respectively. The theoretical framework consists of three steps. First, the vehicle nonholonomic constraint is applied to pairwise feature matching. For vehicle dynamic analysis, the static inliers are selected by choosing the features with their histogram bins consistent with inertial orientations. Second, for the surrounding environment dynamic analysis, the dynamic inliers are matched through histogram voting, together with the developed part-based vehicle detection model that can segment and match the vehicle regions from the background in image pairs. Finally, both the vehicle dynamics and surrounding environments are analyzed with static and dynamic inliers respectively. The proposed method has been evaluated on the challenging datasets, part of which was collected during rush hours in downtown areas. The experimental results prove the effectiveness and accuracy of the proposed VSEDA. INDEX TERMS Vehicle and surrounding environment dynamic analysis (VSEDA), assistive driving, monocular camera, inertial measurement unit (IMU), multi-sensor fusion, complex road conditions.

show abstract

“…1. In order to obtain image plane of virtual camera, a homography-based approach described in [5] has been used which fuses inertial data from IS and image plane of real camera to produce the corresponding virtual cameras image plane.…”

Section: A 3d Data Registrationmentioning

confidence: 99%

A Probabilistic Fusion Framework for 3-D Reconstruction Using Heterogeneous Sensors

et al. 2017

Self Cite

View full text Add to dashboard Cite

This letter proposes a framework to perform 3D reconstruction using a heterogeneous sensor network, with potential use in augmented reality (AR), human behavior understanding, smart-room implementations, robotics, and many other applications. We fuse orientation measurements from inertial sensors, images from cameras and depth data from Time of Flight (ToF) sensors within a probabilistic framework in a synergistic manner to obtain robust reconstructions. A fully probabilistic method is proposed to efficiently fuse the multi-modal data of the system.

show abstract

Heterogeneous Multi-View Information Fusion: Review of 3-D Reconstruction Methods and a New Registration with Uncertainty Modeling

Cited by 25 publications

References 65 publications

Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction

Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction

Robust Vehicle and Surrounding Environment Dynamic Analysis for Assistive Driving Using Visual-Inertial Measurements

A Probabilistic Fusion Framework for 3-D Reconstruction Using Heterogeneous Sensors

Contact Info

Product

Resources

About