A Real-Time Marker-Based Visual Sensor Based on a FPGA and a Soft Core Processor

Tayara, Hilal; Ham, Woon Chul; Chong, Kil To

doi:10.3390/s16122139

Cited by 11 publications

(13 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Recently, many effective AR virtual-real registration methods, which are based on visual fiducial markers [34][35][36], require controllable environments (usually indoor environments) and pre-placed markers. However, in outdoor environments, it is difficult to cover such an uncontrolled environment with markers.…”

Section: Outdoor Augmented Realitymentioning

confidence: 99%

AE-GAN-Net: Learning Invariant Feature Descriptor to Match Ground Camera Images and a Large-Scale 3D Image-Based Point Cloud for Outdoor Augmented Reality

et al. 2019

View full text Add to dashboard Cite

Establishing the spatial relationship between 2D images captured by real cameras and 3D models of the environment (2D and 3D space) is one way to achieve the virtual–real registration for Augmented Reality (AR) in outdoor environments. In this paper, we propose to match the 2D images captured by real cameras and the rendered images from the 3D image-based point cloud to indirectly establish the spatial relationship between 2D and 3D space. We call these two kinds of images as cross-domain images, because their imaging mechanisms and nature are quite different. However, unlike real camera images, the rendered images from the 3D image-based point cloud are inevitably contaminated with image distortion, blurred resolution, and obstructions, which makes image matching with the handcrafted descriptors or existing feature learning neural networks very challenging. Thus, we first propose a novel end-to-end network, AE-GAN-Net, consisting of two AutoEncoders (AEs) with Generative Adversarial Network (GAN) embedding, to learn invariant feature descriptors for cross-domain image matching. Second, a domain-consistent loss function, which balances image content and consistency of feature descriptors for cross-domain image pairs, is introduced to optimize AE-GAN-Net. AE-GAN-Net effectively captures domain-specific information, which is embedded into the learned feature descriptors, thus making the learned feature descriptors robust against image distortion, variations in viewpoints, spatial resolutions, rotation, and scaling. Experimental results show that AE-GAN-Net achieves state-of-the-art performance for image patch retrieval with the cross-domain image patch dataset, which is built from real camera images and the rendered images from 3D image-based point cloud. Finally, by evaluating virtual–real registration for AR on a campus by using the cross-domain image matching results, we demonstrate the feasibility of applying the proposed virtual–real registration to AR in outdoor environments.

show abstract

Section: Outdoor Augmented Realitymentioning

confidence: 99%

AE-GAN-Net: Learning Invariant Feature Descriptor to Match Ground Camera Images and a Large-Scale 3D Image-Based Point Cloud for Outdoor Augmented Reality

et al. 2019

View full text Add to dashboard Cite

show abstract

“…FAST compares surrounding pixels to obtain key points using machine learning. Therefore, it is simple, effective and easily ported to embedded systems [ 18 ]. Calonder et al [ 19 ] proposed the BRIEF descriptor by comparing the PCA, LDA and other feature dimensional reduction methods.…”

Section: Related Workmentioning

confidence: 99%

A novel image registration approach via combining local features and geometric invariants

Gao

Zhang

et al. 2018

PLoS ONE

View full text Add to dashboard Cite

Image registration is widely used in many fields, but the adaptability of the existing methods is limited. This work proposes a novel image registration method with high precision for various complex applications. In this framework, the registration problem is divided into two stages. First, we detect and describe scale-invariant feature points using modified computer vision-oriented fast and rotated brief (ORB) algorithm, and a simple method to increase the performance of feature points matching is proposed. Second, we develop a new local constraint of rough selection according to the feature distances. Evidence shows that the existing matching techniques based on image features are insufficient for the images with sparse image details. Then, we propose a novel matching algorithm via geometric constraints, and establish local feature descriptions based on geometric invariances for the selected feature points. Subsequently, a new price function is constructed to evaluate the similarities between points and obtain exact matching pairs. Finally, we employ the progressive sample consensus method to remove wrong matches and calculate the space transform parameters. Experimental results on various complex image datasets verify that the proposed method is more robust and significantly reduces the rate of false matches while retaining more high-quality feature points.

show abstract

“…Recently, FPGAs have become a viable target technology for implementation different algorithm in remote sensing such us: A Real-Time Marker-Based Visual Sensor Based on a FPGA and a Soft-Core Processor [7], FPGA Implementation of an Algorithm for Automatically Detecting Targets [8], FPGA Implementation of the N-FINDR Algorithm [9], On-Board Ortho-Rectification for images Based on an FPGA [10], FPGA Implementation of JPEG-LS Remote Sensing Image Coding Algorithm [11]. These computing systems combine the flexibility of general-purpose processors with the speed of application-specific processors.…”

Section: Introductionmentioning

confidence: 99%

An approch for FPGA implementation of LST-SW algorithm

Makhloufi¹,

Adib²,

Raissouni³

2019

Proceedings of the Third International Conference on Computing and Wireless Communication Systems, ICCWCS 2019, April 24-25, 20

View full text Add to dashboard Cite

Satellite remote sensing (SRS) needs to make use of up-to-date technology. The trend in SRS missions has always been towards using hardware devices with smaller size, lower cost, more flexibility, and higher computational power. Therefore, Field Programmable Gate Array (FPGA) technology is highly used by the scientific community and hardware developers to implement different SRS algorithms. Thus, FPGAs offer well-suited architectural features; large number of programmable logic elements, distributed and block RAMs, DSP and register slices and look up tables. This paper describes an approach to the implementation of the Land Surface Temperature Split-Window (LST-SW) algorithm structure based on FPGA technology.

show abstract

A Real-Time Marker-Based Visual Sensor Based on a FPGA and a Soft Core Processor

Cited by 11 publications

References 42 publications

AE-GAN-Net: Learning Invariant Feature Descriptor to Match Ground Camera Images and a Large-Scale 3D Image-Based Point Cloud for Outdoor Augmented Reality

AE-GAN-Net: Learning Invariant Feature Descriptor to Match Ground Camera Images and a Large-Scale 3D Image-Based Point Cloud for Outdoor Augmented Reality

A novel image registration approach via combining local features and geometric invariants

An approch for FPGA implementation of LST-SW algorithm

Contact Info

Product

Resources

About