Deep Surface Light Fields

Chen, Anpei; Wu, Minye; Zhang, Yingliang; Li, Nianyi; Lu, Jie; Gao, Shenghua; Yu, Jingyi

doi:10.1145/3203192

Cited by 58 publications

(46 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These methods reconstruct the surface geometry and represent appearance on the surface using lumispheres. Because of the highdimensionality of this representation, methods have focused on compressing this data using PCA and vector quantization [Wood et al 2000] or deep networks [Chen et al 2018]. Our work is able to capture similar appearance effects with vastly fewer images.…”

Section: Related Workmentioning

confidence: 99%

Deep view synthesis from sparse photometric images

Sunkavalli

et al. 2019

ACM Trans. Graph.

View full text Add to dashboard Cite

The goal of light transport acquisition is to take images from a sparse set of lighting and viewing directions, and combine them to enable arbitrary relighting with changing view. While relighting from sparse images has received significant attention, there has been relatively less progress on view synthesis from a sparse set of "photometric" images---images captured under controlled conditions, lit by a single directional source; we use a spherical gantry to position the camera on a sphere surrounding the object. In this paper, we synthesize novel viewpoints across a wide range of viewing directions (covering a 60° cone) from a sparse set of just six viewing directions. While our approach relates to previous view synthesis and image-based rendering techniques, those methods are usually restricted to much smaller baselines, and are captured under environment illumination. At our baselines, input images have few correspondences and large occlusions; however we benefit from structured photometric images. Our method is based on a deep convolutional network trained to directly synthesize new views from the six input views. This network combines 3D convolutions on a plane sweep volume with a novel per-view per-depth plane attention map prediction network to effectively aggregate multi-view appearance. We train our network with a large-scale synthetic dataset of 1000 scenes with complex geometry and material properties. In practice, it is able to synthesize novel viewpoints for captured real data and reproduces complex appearance effects like occlusions, view-dependent specularities and hard shadows. Moreover, the method can also be combined with previous relighting techniques to enable changing both lighting and view, and applied to computer vision problems like multiview stereo from sparse image sets.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep view synthesis from sparse photometric images

Sunkavalli

et al. 2019

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…From yet another angle, ULR-inspired IBR creates a LF (viewdependent appearance) on the surface of a proxy geometry, i.e., a surface light field. Chen et al [2018], using an MLP, as well as Thies et al [2019], using a CNN, have proposed to represent this information using a NN defined in texture space of a proxy object. While inspired by the mechanics of sparse IBR, results are typically demonstrated for rather dense observations.…”

Section: View Interpolation (Light Fields)mentioning

confidence: 99%

X-Fields

et al. 2020

View full text Add to dashboard Cite

We suggest to represent an X-Field ---a set of 2D images taken across different view, time or illumination conditions, i.e., video, lightfield, reflectance fields or combinations thereof---by learning a neural network (NN) to map their view, time or light coordinates to 2D images. Executing this NN at new coordinates results in joint view, time or light interpolation. The key idea to make this workable is a NN that already knows the "basic tricks" of graphics (lighting, 3D projection, occlusion) in a hard-coded and differentiable form. The NN represents the input to that rendering as an implicit map, that for any view, time, or light coordinate and for any pixel can quantify how it will move if view, time or light coordinates change (Jacobian of pixel position with respect to view, time, illumination, etc.). Our X-Field representation is trained for one scene within minutes, leading to a compact set of trainable parameters and hence real-time navigation in view, time and illumination.

show abstract

“…Anpei et al [15] introduced a novel algorithm based on the neural network called "Deep Surface Light Field" or DSLF for moderate sampling. Leveraging different patterns of sampling, DSLF fills in the missing data.…”

Section: Literature Reviewmentioning

confidence: 99%

Detecting Video Surveillance Using VGG19 Convolutional Neural Networks

Butt¹,

Letchmunan²,

Hafinaz³

et al. 2020

IJACSA

View full text Add to dashboard Cite

The meteoric growth of data over the internet from the last few years has created a challenge of mining and extracting useful patterns from a large dataset. In recent years, the growth of digital libraries and video databases makes it more challenging and important to extract useful information from raw data to prevent and detect the crimes from the database automatically. Street crime snatching and theft detection is the major challenge in video mining. The main target is to select features/objects which usually occurs at the time of snatching. The number of moving targets imitates the performance, speed and amount of motion in the anomalous video. The dataset used in this paper is Snatch 101; the videos in the dataset are further divided into frames. The frames are labelled and segmented for training. We applied the VGG19 Convolutional Neural Network architecture algorithm and extracted the features of objects and compared them with original video features and objects. The main contribution of our research is to create frames from the videos and then label the objects. The objects are selected from frames where we can detect anomalous activities. The proposed system is never used before for crime prediction, and it is computationally efficient and effective as compared to state-of-the-art systems. The proposed system outperformed with 81 % accuracy as compared to stateof-the-art systems.

show abstract

Deep Surface Light Fields

Cited by 58 publications

References 26 publications

Deep view synthesis from sparse photometric images

Deep view synthesis from sparse photometric images

X-Fields

Detecting Video Surveillance Using VGG19 Convolutional Neural Networks

Contact Info

Product

Resources

About