Masked photo blending: Mapping dense photographic data set on high-resolution sampled 3D models

Abstract

The technological advance of sensors is producing an exponential size growth of the data coming from 3D scanning and digital photography. The production of digital 3D models consisting of tens or even hundreds of millions of triangles is quite easy nowadays; at the same time, using high-resolution digital cameras it is also straightforward to produce a set of pictures of the same real object totalling more than 50M Pixel.The problem is how to manage all this data to produce 3D models that could fit the interactive rendering constraints. A common approach is to go for mesh parametrization and texture synthesis, but finding a parametrization for such large meshes and managing such large textures can be prohibitive. Moreover, digital photo sampling produces highly redundant data; this redundancy should be eliminated while mapping to the 3D model but, at the same time, should also be efficiently used to improve the sampled data coherence and the appearance representation accuracy.In this paper we present an approach where a multivariate blending function weights all the available pixel data with respect to geometric, topological and colorimetric criteria. The blending approach proposed is efficient, since it mostly works independently on each image, and can be easily extended to include other image quality estimators. The resulting weighted pixels are then selectively mapped on the geometry, preferably by adopting a multiresolution per-vertex encoding to make profitable use of all the data available and to avoid the texture size bottleneck. Some practical examples on complex datasets are presented.

From the Digitization of Cultural Artifacts to the Web Publishing of Digital 3D Collections: an Automatic Pipeline for Knowledge Sharing

“…Our texturing method extends the work proposed in [20] so as to adapt it to the data flow produced by the scanning system presented above. The idea, summarized in Figure 3, is to weight each input picture by a mask (typically a gray scale image) which represents a per-pixel confidence value.…”

“…The weight is usually a combination of various quality metrics [17]- [19]. In particular, Callieri et al [20] presented a flexible weighting system that can be extended in order to accommodate additional criteria. These methods provide better visual results and their implementation permits very complex datasets to be used, i.e.…”

Heritage Recording and 3D Modeling with Photogrammetry and 3D Scanning

“…Indeed images are often exposed with the illumination at imaging time, but it may need to be replaced by illumination consistent with the rendering point of view and the reflectance properties (bidirectional reflectance distribution function) of the object [119]. High dynamic range (HDR) images might also be acquired to recover all scene details [120] while color discontinuities and aliasing effects must be removed [121]. Methods for occlusions removal are primarily based on background learning and subtraction, visibility analyses, image rectification and estimation of not-occluded pixels or manual retouch [122][123][124][125].…”

3d Digitization of an Heritage Masterpiece - A Critical Analysis on Quality Assessment

“…8c). The colour projection and integration over the Sarcophagus mesh was applied using the Masked Photo Blending (Callieri et al, 2008) approach. In the case of complex projects, the number of images and the amount of detail to be preserved may pose severe issues.…”