Camera tracking in visual effects an industry perspective of structure from motion

Barber, Alastair; Cosker, Darren; James, Oliver; Waine, Ted; Patel, Radhika

doi:10.1145/2947688.2947697

Cited by 12 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3.4 (e.g tracking and 3D rendering). AI techniques 63 are increasingly being employed to reduce the human resources needed for certain labour-intensive or repetitive tasks such as matchmove, tracking, rotoscoping, compositing and animation (Barber et al 2016;Torrejon et al 2020).…”

Section: Visual Special Effects (Vfx)mentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

This paper reviews the current state of the art in artificial intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically machine learning (ML) algorithms, is provided including convolutional neural networks (CNNs), generative adversarial networks (GANs), recurrent neural networks (RNNs) and deep Reinforcement Learning (DRL). We categorize creative applications into five groups, related to how AI technologies are used: (i) content creation, (ii) information analysis, (iii) content enhancement and post production workflows, (iv) information extraction and enhancement, and (v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, ML-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of ML in domains with fewer constraints, where AI is the ‘creator’, remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human-centric—where it is designed to augment, rather than replace, human creativity.

show abstract

Section: Visual Special Effects (Vfx)mentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

show abstract

“…The motivation for our paper was to find out whether the presence of a background can improve the quality of a model of a given object. To answer this question, we reconstructed the same objects in a classical SfM pipeline COLMAP [49] 4 , which is considered to be the state-of-theart method. COLMAP cannot work with multiple objects, therefore, we run COLMAP on a different dataset 5 with a similar number of images, where the object is depicted without background.…”

Section: Comparison With Single Body Sfmmentioning

confidence: 99%

“…SfM is well understood for static scenes with a single rigid object and has many practical applications in, e.g., cartography [10], archaeology [68], and film industry [4]. In the case of dynamic scenes, the majority of works [15,57,41,48,38,44] and [14,45,24,23,66,16] assume video input, while the case of unordered input images has not been fully explored yet.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reconstructing Small 3D Objects in front of a Textured Background

Hrubý¹,

Pajdla²

2021

Preprint

View full text Add to dashboard Cite

We present a technique for a complete 3D reconstruction of small objects moving in front of a textured background. It is a particular variation of multibody structure from motion, which specializes to two objects only. The scene is captured in several static configurations between which the relative pose of the two objects may change. We reconstruct every static configuration individually and segment the points locally by finding multiple poses of cameras that capture the scene's other configurations. Then, the local segmentation results are combined, and the reconstructions are merged into the resulting model of the scene. In experiments with real artifacts, we show that our approach has practical advantages when reconstructing 3D objects from all sides. In this setting, our method outperforms the stateof-the-art. We integrate our method into the state of the art 3D reconstruction pipeline COLMAP.

show abstract

“…Our approach synthesizes object detection with SfM into a coherent framework that can be deployed in a variety of systems without domain-specific expertise. SfM is commonly used for 3D environmental reconstructions, photogrammetry and camera tracking for visual effects in video editing [32,33], and here allows the reconstruction of 3D models of the terrain through which the animals move and interact with. Our open-source analysis pathway enables subsequent calculation of movement, interactions, and postures of animals.…”

mentioning

confidence: 99%

High-resolution, non-invasive animal tracking and reconstruction of local environment in aquatic ecosystems

2020

View full text Add to dashboard Cite

Background: Acquiring high resolution quantitative behavioural data underwater often involves installation of costly infrastructure, or capture and manipulation of animals. Aquatic movement ecology can therefore be limited in taxonomic range and ecological coverage. Methods: Here we present a novel deep-learning based, multi-individual tracking approach, which incorporates Structure-from-Motion in order to determine the 3D location, body position and the visual environment of every recorded individual. The application is based on low-cost cameras and does not require the animals to be confined, manipulated, or handled in any way. Results: Using this approach, single individuals, small heterospecific groups and schools of fish were tracked in freshwater and marine environments of varying complexity. Positional tracking errors as low as 1.09 ± 0.47 cm (RSME) in underwater areas up to 500 m 2 were recorded. Conclusions: This cost-effective and open-source framework allows the analysis of animal behaviour in aquatic systems at an unprecedented resolution. Implementing this versatile approach, quantitative behavioural analysis can be employed in a wide range of natural contexts, vastly expanding our potential for examining non-model systems and species.

show abstract

Camera tracking in visual effects an industry perspective of structure from motion

Cited by 12 publications

References 22 publications

Artificial intelligence in the creative industries: a review

Artificial intelligence in the creative industries: a review

Reconstructing Small 3D Objects in front of a Textured Background

High-resolution, non-invasive animal tracking and reconstruction of local environment in aquatic ecosystems

Contact Info

Product

Resources

About