Towards real-time photorealistic 3D holography with deep neural networks

Shi, Liang; Li, Beichen; Kim, Chang-Il; Kellnhofer, Petr; Matusik, Wojciech

doi:10.1038/s41586-020-03152-0

Cited by 407 publications

(210 citation statements)

References 51 publications

Supporting

Mentioning

209

Contrasting

Order By: Relevance

“…The former usually uses artificially designed features to detect key points. During this period, the commonly used feature extraction methods include HOG, shape content descriptor [ 2 ], and multimethod synthesis [ 3 ]; the tree structure model is used to model recognition features [ 4 – 6 ]. In the period of deep learning, because of the improvement of computer computing ability, the convolutional neural network as the representative of deep learning develops rapidly, so it is often used in image feature extraction.…”

Section: Related Workmentioning

confidence: 99%

[Retracted] Human Skeleton Detection and Extraction in Dance Video Based on PSO‐Enabled LSTM Neural Network

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

With the significant increase of social informatization, the emerging information technology represented by machine vision has been applied to more and more scenes. Among them, the detection and extraction of human skeleton in a dance video based on this technology has a huge market demand in education and training. However, the existing detection and extraction technology has the problems of slow recognition speed and low extraction accuracy. Therefore, this paper proposes a neural network based on particle swarm optimization to detect and extract human skeletons in a dance video. Through the research and test on different data sets, it is found that the neural network based on particle swarm optimization algorithm has good detection and extraction ability and has high accuracy for the detection and recognition of human skeleton points. Among them, on all MPII data sets, the average accuracy of PSO-LSTM proposed in this paper is 3.9% higher than that of other optimal algorithms; on the PoseTrack data set, the average accuracy of detection and extraction is improved by 2.3%. The above results show that the neural network based on particle swarm optimization has fast detection speed and good extraction accuracy and can be used for the detection and extraction of human skeleton in a dance video.

show abstract

Section: Related Workmentioning

confidence: 99%

[Retracted] Human Skeleton Detection and Extraction in Dance Video Based on PSO‐Enabled LSTM Neural Network

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…With the development of the deep learning algorithm, this algorithm is gradually combined with artificial intelligence devices [ 21 ]. It is mainly reflected in the deep learning of the DBN algorithm and backpropagation neural network combined with optimization, forming the ability to analyse the handwriting similarity and deal with fuzzy picture handwriting and other functions.…”

Section: Related Workmentioning

confidence: 99%

[Retracted] Research on Information Visualization Graphic Design Teaching Based on DBN Algorithm

Xue

2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

With the advent of the era of big data, how to quickly obtain effective information and efficiently disseminate information technology has become the most popular topic. Studies have shown that the ability of the human brain to process data and information is unmatched by machines, and the processing of graphics is tens of thousands of times faster than that of words. Based on the deep belief network (DBN) algorithm, this paper studies the technology of information visualization graphic design teaching application. Firstly, the structure of the deep belief network is analysed to explore its technical application in graphic information reconstruction. It is concluded that the DBN algorithm can be used to deal with the problems of classification, regression, dimension calculation, feature point acquisition, accuracy calculation, and so on in machine learning training. Then, the deformation technology of graphic local design is studied based on the DBN algorithm to construct the visual teaching platform and analyse the technical research results of this algorithm in information graphic design. The results show that the DBN algorithm can quickly solve the problem of processing complex features in graphics, change the local deformation design of the original graphics to form new feature point data and add it to the teaching platform, and improve the ability of model fast learning and training, optimizing the operation efficiency of the teaching platform.

show abstract

“…Therefore, it is an excellent choice for fast and efficient processing of optical information (including fast CGH generation). Neural networks have also been successfully applied to digital holography [16], computational imaging [17][18][19], hologram generation [20][21][22][23][24], etc.…”

Section: Introductionmentioning

confidence: 99%

Computing 3D Phase-Type Holograms Based on Deep Learning Method

Zheng

Zhou

et al. 2021

Photonics

View full text Add to dashboard Cite

Computer holography is a technology that use a mathematical model of optical holography to generate digital holograms. It has wide and promising applications in various areas, especially holographic display. However, traditional computational algorithms for generation of phase-type holograms based on iterative optimization have a built-in tradeoff between the calculating speed and accuracy, which severely limits the performance of computational holograms in advanced applications. Recently, several deep learning based computational methods for generating holograms have gained more and more attention. In this paper, a convolutional neural network for generation of multi-plane holograms and its training strategy is proposed using a multi-plane iterative angular spectrum algorithm (ASM). The well-trained network indicates an excellent ability to generate phase-only holograms for multi-plane input images and to reconstruct correct images in the corresponding depth plane. Numerical simulations and optical reconstructions show that the accuracy of this method is almost the same with traditional iterative methods but the computational time decreases dramatically. The result images show a high quality through analysis of the image performance indicators, e.g., peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and contrast ratio. Finally, the effectiveness of the proposed method is verified through experimental investigations.

show abstract

Towards real-time photorealistic 3D holography with deep neural networks

Cited by 407 publications

References 51 publications

[Retracted] Human Skeleton Detection and Extraction in Dance Video Based on PSO‐Enabled LSTM Neural Network

[Retracted] Human Skeleton Detection and Extraction in Dance Video Based on PSO‐Enabled LSTM Neural Network

[Retracted] Research on Information Visualization Graphic Design Teaching Based on DBN Algorithm

Computing 3D Phase-Type Holograms Based on Deep Learning Method

Contact Info

Product

Resources

About