Investigating 3D holoscopic visual content upsampling using super-resolution for cultural heritage digitization

Belhi, Abdelhak; Bouras, Abdelaziz; Alfaqheri, Taha; Aondoakaa, Akuha Solomon; Sadka, A.H.

doi:10.1016/j.image.2019.04.005

Cited by 13 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nowadays, a lot of challenges are presented to data scientists and artificial intelligence professionals to annotate cultural objects through data classification techniques, image reconstruction and semantic web technologies. In the following sections, we present two new approaches intended for the enrichment of cultural heritage assets with the use of data classification and image reconstruction (Van Noord et al , 2015; Belhi et al , 2019; Krizhevsky et al , 2012).…”

Section: Digital Cultural Heritagementioning

confidence: 99%

“…The tendency changed since the rise of deep learning with computer vision and natural language processing applications and multiple applications were proposed in the cultural context. The challenge is how to use the power of these tools to effectively solve problems in the cultural context (Van Noord et al , 2015; Belhi et al , 2019). Through this paper, we present two new approaches focusing on the annotation and classification of cultural heritage assets along with the reconstruction of their damaged visual information.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A machine learning framework for enhancing digital experiences in cultural heritage

Belhi

Bouras

Al‐Ali

et al. 2020

JEIM

View full text Add to dashboard Cite

PurposeDigital tools have been used to document cultural heritage with high-quality imaging and metadata. However, some of the historical assets are totally or partially unlabeled and some are physically damaged, which decreases their attractiveness and induces loss of value. This paper introduces a new framework that aims at tackling the cultural data enrichment challenge using machine learning.Design/methodology/approachThis framework focuses on the automatic annotation and metadata completion through new deep learning classification and annotation methods. It also addresses issues related to physically damaged heritage objects through a new image reconstruction approach based on supervised and unsupervised learning.FindingsThe authors evaluate approaches on a data set of cultural objects collected from various cultural institutions around the world. For annotation and classification part of this study, the authors proposed and implemented a hierarchical multimodal classifier that improves the quality of annotation and increases the accuracy of the model, thanks to the introduction of multitask multimodal learning. Regarding cultural data visual reconstruction, the proposed clustering-based method, which combines supervised and unsupervised learning is found to yield better quality completion than existing inpainting frameworks.Originality/valueThis research work is original in sense that it proposes new approaches for the cultural data enrichment, and to the authors’ knowledge, none of the existing enrichment approaches focus on providing an integrated framework based on machine learning to solve current challenges in cultural heritage. These challenges, which are identified by the authors are related to metadata annotation and visual reconstruction.

show abstract

Section: Digital Cultural Heritagementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A machine learning framework for enhancing digital experiences in cultural heritage

Belhi

Bouras

Al‐Ali

et al. 2020

JEIM

View full text Add to dashboard Cite

show abstract

“…The performance of such light field systems, however, has been limited by the camera sensor size (number of pixels) and of the characteristics of the microlens array [18,19]. These two factors are calculated according to the sampling theorem to optimise the trade-off results between spatial and angular resolution of the recorded light field image [18].…”

Section: Introductionmentioning

confidence: 99%

Low-delay single holoscopic 3D computer-generated image to multiview images

Alfaqheri

Aondoakaa

Swash

et al. 2020

J Real-Time Image Proc

Self Cite

View full text Add to dashboard Cite

Due to the nature of holoscopic 3D (H3D) imaging technology, H3D cameras can capture more angular information than their conventional 2D counterparts. This is mainly attributed to the macrolens array which captures the 3D scene with slightly different viewing angles and generates holoscopic elemental images based on fly’s eyes imaging concept. However, this advantage comes at the cost of decreasing the spatial resolution in the reconstructed images. On the other hand, the consumer market is looking to find an efficient multiview capturing solution for the commercially available autostereoscopic displays. The autostereoscopic display provides multiple viewers with the ability to simultaneously enjoy a 3D viewing experience without the need for wearing 3D display glasses. This paper proposes a low-delay content adaptation framework for converting a single holoscopic 3D computer-generated image into multiple viewpoint images. Furthermore, it investigates the effects of varying interpolation step sizes on the converted multiview images using the nearest neighbour and bicubic sampling interpolation techniques. In addition, it evaluates the effects of changing the macrolens array size, using the proposed framework, on the perceived visual quality both objectively and subjectively. The experimental work is conducted on computer-generated H3D images with different macrolens sizes. The experimental results show that the proposed content adaptation framework can be used to capture multiple viewpoint images to be visualised on autostereoscopic displays.

show abstract

“…However, the physical preservation of these assets is a tedious and delicate process that requires a lot of time, resources and needs to be undertaken by highly skilled professional curators. As a cost-effective and reliable way for art and culture preservation, digital technologies offer additional ways to preserve and further give more value and excitement around cultural heritage [1][2][3]. A lot of effort was undertaken to provide Information Technology (IT) solutions in the cultural domain.…”

Section: Introductionmentioning

confidence: 99%

Machine Learning and Digital Heritage: The CEPROQHA Project Perspective

Belhi¹,

Gasmi²,

Bouras³

et al. 2020

Advances in Intelligent Systems and Computing

Self Cite

View full text Add to dashboard Cite

Museums and cultural institutions, in general, are in a constant challenge of adding more value to their collections. The attractiveness of assets is practically tightly related to their value obeying the offer and demand law. New digital visualization technologies are found to give more excitements, especially to the younger generation as it is proven by multiple studies. Nowadays, museums around the world are currently trying to promote their collections through new multimedia and digital technologies such as 3D modeling, Virtual Reality (VR), Augmented Reality (AR), serious games, etc. However, the difficulty and the resources required to implement such technologies present a real challenge. Through this paper, we propose a 3D acquisition and visualization framework aiming mostly at increasing the value of cultural collections. This framework preserves cost-effectiveness and time constraints while still introducing new ways of visualization and interaction with high-quality 3D models of cultural objects.

show abstract

Investigating 3D holoscopic visual content upsampling using super-resolution for cultural heritage digitization

Cited by 13 publications

References 34 publications

A machine learning framework for enhancing digital experiences in cultural heritage

A machine learning framework for enhancing digital experiences in cultural heritage

Low-delay single holoscopic 3D computer-generated image to multiview images

Machine Learning and Digital Heritage: The CEPROQHA Project Perspective

Contact Info

Product

Resources

About