RGB-D-Based Face Reconstruction and Recognition

Hsu, Gee-Sern; Liu, Yu-Lun; Peng, Hsiao-Chia; Wu, Po-Xun

doi:10.1109/tifs.2014.2361028

Cited by 31 publications

(8 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using multiple frames would allow for exploration of what improvements may be achieved with the use of recurrent neural networks (RNN) for they have shown to be capable of predicting sequential data [74][75][76]. Finally, using the RGB frames combined with depth frames for reconstruction can potentially add some missing features from the depth due to inherent noisiness of the sensor, therefore improving the recall rate [77,78].…”

Section: Discussionmentioning

confidence: 99%

3D Object Reconstruction from Imperfect Depth Data Using Extended YOLOv3 Network

Kulikajevas

Maskeliūnas

Damaševičius

et al. 2020

Sensors

View full text Add to dashboard Cite

State-of-the-art intelligent versatile applications provoke the usage of full 3D, depth-based streams, especially in the scenarios of intelligent remote control and communications, where virtual and augmented reality will soon become outdated and are forecasted to be replaced by point cloud streams providing explorable 3D environments of communication and industrial data. One of the most novel approaches employed in modern object reconstruction methods is to use a priori knowledge of the objects that are being reconstructed. Our approach is different as we strive to reconstruct a 3D object within much more difficult scenarios of limited data availability. Data stream is often limited by insufficient depth camera coverage and, as a result, the objects are occluded and data is lost. Our proposed hybrid artificial neural network modifications have improved the reconstruction results by 8.53% which allows us for much more precise filling of occluded object sides and reduction of noise during the process. Furthermore, the addition of object segmentation masks and the individual object instance classification is a leap forward towards a general-purpose scene reconstruction as opposed to a single object reconstruction task due to the ability to mask out overlapping object instances and using only masked object area in the reconstruction process.

show abstract

Section: Discussionmentioning

confidence: 99%

3D Object Reconstruction from Imperfect Depth Data Using Extended YOLOv3 Network

Kulikajevas

Maskeliūnas

Damaševičius

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Moreover, additional functionality in the method such as solving homography would allow us to extract the transformation matrix of the object, allowing the system to be used for such applications as Virtual Reality in conjunction with Augmented Reality. Finally, using RGB sensor frames in conjunction with depth frames may add some missing features to improve the recall rate even more [50,51].…”

Section: Discussion and Concluding Remarksmentioning

confidence: 99%

Reconstruction of 3D Object Shape Using Hybrid Modular Neural Network Architecture Trained on 3D Models from ShapeNetCore Dataset

Kulikajevas

Maskeliūnas

Damaševičius

et al. 2019

Sensors

View full text Add to dashboard Cite

Depth-based reconstruction of three-dimensional (3D) shape of objects is one of core problems in computer vision with a lot of commercial applications. However, the 3D scanning for point cloud-based video streaming is expensive and is generally unattainable to an average user due to required setup of multiple depth sensors. We propose a novel hybrid modular artificial neural network (ANN) architecture, which can reconstruct smooth polygonal meshes from a single depth frame, using a priori knowledge. The architecture of neural network consists of separate nodes for recognition of object type and reconstruction thus allowing for easy retraining and extension for new object types. We performed recognition of nine real-world objects using the neural network trained on the ShapeNetCore model dataset. The results evaluated quantitatively using the Intersection-over-Union (IoU), Completeness, Correctness and Quality metrics, and qualitative evaluation by visual inspection demonstrate the robustness of the proposed architecture with respect to different viewing angles and illumination conditions.

show abstract

“…Sequential Methods [29] used a sequence of temporal images to perform 2D face recognition. Recent works [1,17,12, 5] adopted a sequence of 3D data to perform depth fusion or morphology to reconstruct face models. They adopted ICP algorithm to perform registration for point clouds acquired by Kinect.…”

Section: Related Workmentioning

confidence: 99%

Face Recognition from Sequential Sparse 3D Data via Deep Registration

Yang

Lin

Xiao

et al. 2019

2019 International Conference on Biometrics (ICB)

View full text Add to dashboard Cite

Previous works have shown that face recognition with high accurate 3D data is more reliable and insensitive to pose and illumination variations. Recently, low-cost and portable 3D acquisition techniques like ToF(Time of Flight) and DoE based structured light systems enable us to access 3D data easily, e.g., via a mobile phone. However, such devices only provide sparse(limited speckles in structured light system) and noisy 3D data which can not support face recognition directly. In this paper, we aim at achieving high-performance face recognition for devices equipped with such modules which is very meaningful in practice as such devices will be very popular. We propose a framework to perform face recognition by fusing a sequence of low-quality 3D data. As 3D data are sparse and noisy which can not be well handled by conventional methods like the ICP algorithm, we design a PointNet-like Deep Registration Network(DRNet) which works with ordered 3D point coordinates while preserving the ability of mining local structures via convolution. Meanwhile we develop a novel loss function to optimize our DRNet based on the quaternion expression which obviously outperforms other widely used functions. For face recognition, we design a deep convolutional network which takes the fused 3D depth-map as input based on AMSoftmax model. Experiments show that our DRNet can achieve rotation error 0.95 • and translation error 0.28mm for registration. The face recognition on fused data also achieves rank-1 accuracy 99.2% , FAR-0.001 97.5% on Bosphorus dataset which is comparable with state-of-the-art high-quality data based recognition performance.

show abstract

RGB-D-Based Face Reconstruction and Recognition

Cited by 31 publications

References 21 publications

3D Object Reconstruction from Imperfect Depth Data Using Extended YOLOv3 Network

3D Object Reconstruction from Imperfect Depth Data Using Extended YOLOv3 Network

Reconstruction of 3D Object Shape Using Hybrid Modular Neural Network Architecture Trained on 3D Models from ShapeNetCore Dataset

Face Recognition from Sequential Sparse 3D Data via Deep Registration

Contact Info

Product

Resources

About