基于神经辐射场的光场角度域超分辨

Abstract

Objective In response to the tradeoff of spatialangular resolution in light field data acquisition due to data flux limitations, we propose a neural radiance fieldbased method to achieve highquality light field superresolution in the angular domain. The occlusion, depth variations, and background interference make superresolution in the angular domain a challenging task, and it is difficult to express the rich details of the texture. In order to address these issues, many solutions are proposed in terms of novel view synthesis based on explicit and implicit scene geometry. However, both explicit and implicit scene geometry methods generate new viewpoint images of the scene from the geometric features of the scene, which are prone to problems such as noise interference and difficult reconstruction of textural details.Therefore, we propose neural radiance fieldbased light field superresolution in the angular domain to reconstruct densely sampled light fields from sparse viewpoint sets, which can avoid errors and noises that may be introduced during image acquisition and improve the accuracy and quality of subsequent threedimensional (3D) reconstruction.Methods By training the neural network with the light field data, the neural radiance field captures the complete scene information, even for novel viewpoints, and thus enhances the scene representation performance. In order to achieve this, a multilayer perceptron is utilized to express a fivedimensional vector function that describes the geometry and color information of the 3D model. The image color is then predicted using volume rendering. The light field is subsequently represented by a neural radiance field, and dense sampling of the angular dimension is achieved by adjusting the camera pose in the light field to obtain new perspectives between the subaperture images. This approach overcomes the limitations of prior techniques, including occlusion, depth variation, and background interference in light field scenes.Additionally, the input variable is mapped to the Fourier features of that variable by positional encoding, effectively addressing the challenge of fitting to the highfrequency textural information of the scene. Results and DiscussionsWe propose the neural radiance fieldbased light field superresolution in the angular domain by representing the light field by the neural radiance field. The main advantage of the proposed method over the selected experimental methods, such as local light field fusion (LLFF) and light field reconstruction using convolutional network on EPI (LFEPICNN) is that the proposed method is based on the neural radiance field to implicitly represent the light field scene, which can fit an accurate implicit function for the highresolution fourdimensional light field and accurately represent the light field scene with complex conditions. The experimental results show that the superresolution method based on the neural radiance field proposed can improve the angular resolution from 5×5 to 9×9. The peak signal to noise ratio (PSNR) is ...