Material recognition using time of flight Lidar surface analysis

Tafone, Daniel; McEnvoy, Luke; Sua, Yong Meng; Rehain, Patrick; Huang, Yu‐Ping

doi:10.1117/12.2652945

Cited by 4 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, Ultrasonic-based methods [14] operate by measuring the grain size of the target material. On the other hand, Time of Flight (ToF) sensor methods, including ToF [15] cameras and ToF LiDARs [16], identify materials by assessing surface roughness by the speed of the reflected wave. Despite their potential, these methods necessitate precise scanning aiming at the material of interest, rendering them unsuitable for non-constrained environments where such precision may be unattainable.…”

Section: Introductionmentioning

confidence: 99%

WITHDRAWN: Surface multi-spectral reflectivity and texture material recognition in non-constrained environments

Dhahri,

Amamou,

Faghihi

et al. 2024

Preprint

View full text Add to dashboard Cite

Material recognition is the process of distinguishing various materials based on their inherent physical properties. It plays a pivotal role in numerous applications, including manufacturing, recycling, and robotic handling. Conventional recognition methods predominantly employ sensor-and vision-based approaches. However, these methods often face challenges such as the similarity and variability in material appearance, environmental conditions, and geometric constraints. In this research, we introduce a multimodal, vision-based, attention-driven model for material recognition. Contrary to preceding texture-based and multi-spectral based methods, our approach harnesses both the texture and light reflection distribution characteristics intrinsic to material surfaces. The proposed method features a collocated system that combines depth, RGB, and near-infrared (Near-IR) cameras, along with infrared laser projector. This specific setup was selected to capture reflection distribution and texture across the visible-near-infrared spectrum. Subsequently, the data captured by this setup were processed by a recognition model within a fusion framework. Our results outperform previous methods in terms of accuracy when additional modalities (Depth, Near-IR, laser projectors) are available, while also exhibiting equivalent performance to top RGB-based models when solely reliant on RGB data. Thus, proving the complementarity of the added modalities with visible information.

show abstract

Section: Introductionmentioning

confidence: 99%

WITHDRAWN: Surface multi-spectral reflectivity and texture material recognition in non-constrained environments

Dhahri,

Amamou,

Faghihi

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…It benefits significantly from ultrahigh detection sensitivity on a single-photon level and the ability to time-tag photon arrivals with nanoto-picosecond resolution. Recently, such technology and its derivatives have been deployed in remote sensing for millimeter to kilometer working distances [31][32][33][34][35][36][37] , with interesting applications in fluorescence Spectroscopy 38,39 , astronomy 40,41 , biomedical imaging of cancer and x-rays 42,43 , environmental imaging of photosynthesis and underwater scenes [44][45][46] , bio-metrics for reading heart beats 47,48 and more. In this work, we down-sample an image plane via a random physical mask on the scanning pattern (i.e., scanning only a fraction of pixels) and reconstruct the full image by using PI-MAE that takes both the photon-counting results and the scanning pattern.…”

Section: Introductionmentioning

confidence: 99%

Physics-Informed Masked Autoencoder for Active Sparse Imaging

McEvoy,

Tafone,

Sua

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Imaging technology based on detecting individual photons has seen tremendous progress in recent years, with broad applications in autonomous driving, biomedical imaging, astronomical observation, and more. Comparing with conventional methods, however, it takes much longer time and relies on sparse and noisy photon-counting data to form an image. Here we introduce Physics-Informed Masked Autoencoder (PI-MAE) as a fast and efficient approach for data acquisition and image reconstruction through hardware implementation of the MAE (Masked Autoencoder). We examine its performance on a single-photon LiDAR system when trained on digitally masked MNIST data. Our results show that, with 1.8 × 10−6 or less detected photons per pulse and down to 9 detected photons per pixel, it achieves high-quality image reconstruction on unseen object classes with 90% physical masking. Our results highlight PI-MAE as a viable hardware accelerator for significantly improving the performance of single-photon imaging systems in photon-starving applications.

show abstract

“…Single-photon LiDAR has become one of the most promising technologies for novel imaging and sensing modalities with single-photon sensitivity and picosecond temporal resolution. 1,2,3 Despite its enormous potential, single-photon LiDAR applications are prohibited by the sparsity of signal photons that are mixed with strong background noise and a very limited imaging frame rate. 4 In this paper, we introduce a novel approach to LiDAR data acquisition and reconstruction, employing a dynamic masking technique in conjunction with an inpainting transformer model.…”

Section: Introductionmentioning

confidence: 99%

Inpainting sparse scenes through physics aware transformers for single-photon LiDAR

McEvoy,

Tafone,

Sua

et al. 2024

Unconventional Optical Imaging IV

View full text Add to dashboard Cite

We increase single-photon LiDAR capabilities via a hardware-accelerating inpainting transformer model. This model reconstructs all non-observed information within the image plane as it communicates with the beam steering hardware. We apply this to 3D time-of-flight (ToF) reconstruction, where objects obstruct each other's line of sight. We use ToF histograms to distinguish objects within either the foreground and background, and their overlap will be treated as the dynamic mask for the model to reconstruct. We also employ this to unorthodox scanning patterns such as Lissajous and spiral, which are riddled with sparsity. Lastly, we are developing an AI MEMs system, which intelligently downsamples the image plane based off foreground masks, combating sampling redundancy. We believe that our approach will be useful in applications for imaging and sensing dynamic targets with sparse single-photon data across all domains.

show abstract

Material recognition using time of flight Lidar surface analysis

Cited by 4 publications

References 9 publications

WITHDRAWN: Surface multi-spectral reflectivity and texture material recognition in non-constrained environments

WITHDRAWN: Surface multi-spectral reflectivity and texture material recognition in non-constrained environments

Physics-Informed Masked Autoencoder for Active Sparse Imaging

Inpainting sparse scenes through physics aware transformers for single-photon LiDAR

Contact Info

Product

Resources

About