Fully Convolutional Network and Region Proposal for Instance Identification with Egocentric Vision

Portaz, Maxime; Kohl, Matthias; Quénot, Georges; Chevallet, Jean–Pierre

doi:10.1109/iccvw.2017.281

Cited by 11 publications

(8 citation statements)

References 25 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To acquire the images, cameras can be mounted on smartphones or wearable models provided to the visitors. Artwork image detection has been proposed in [22,23], these approaches require a training phase and are susceptible to partial visual obstruction caused for instance by a crowded environment. In [23], the authors present an approach for image retrieval and object detection based on fully convolutional networks but it is particularly targeted at small datasets with low object variability.…”

Section: Vision-based Techniquesmentioning

confidence: 99%

“…Artwork image detection has been proposed in [22,23], these approaches require a training phase and are susceptible to partial visual obstruction caused for instance by a crowded environment. In [23], the authors present an approach for image retrieval and object detection based on fully convolutional networks but it is particularly targeted at small datasets with low object variability. Moreover, it should be taken into account that museums are complex environments, which might expose plenty of artworks, and each of them requires to be classified from different perspectives, distances and lighting conditions [6].…”

Section: Vision-based Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Detecting Proximity with Bluetooth Low Energy Beacons for Cultural Heritage

Barsocchi

Girolami

Rosa

2021

Sensors

View full text Add to dashboard Cite

The RE.S.I.STO project targets visitors of Pisa medieval city, with the goal of providing high-quality digital contents accessible with smart devices. We describe the design, implementation and the test phases of the RE.S.I.STO application, whose goal is to automatically detect the proximity between visitors and artworks. Proximity is detected with a set of algorithms based on the analysis of Bluetooth Low Energy beacons. We detail our experimental campaigns which reproduce several museum layouts of increasing complexity at two pilot sites, and we compute the performance of the implemented algorithms to detect the nearby artworks. In particular, we test our solution in a wide open space located in our research institute and by performing a real deployment at the Camposanto Monumentale located in Pisa (Italy). The obtained performance varies in the range of 40% to perfect accuracy, according to the complexity of the considered museum layouts. We also describe a set of stress and stability tests aimed at verifying the robustness of the application during the data collection process. Our results show that the mobile application is able to reduce the beacon loss rate, with an average value of 77% of collected beacons.

show abstract

Section: Vision-based Techniquesmentioning

confidence: 99%

Section: Vision-based Techniquesmentioning

confidence: 99%

Detecting Proximity with Bluetooth Low Energy Beacons for Cultural Heritage

Barsocchi

Girolami

Rosa

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Wearable devices can be used to improve the fruition of artworks and the user experience in cultural sites [10]. The authors of [2], [11] propose a smart audio guide that, based on the actions and interests of museum visitors, interacts with the visitors improving their experience and the fruition of multimedia materials. An important ability for these systems is related to the detection of artworks, which can be achieved using object detectors [4], [6].…”

Section: Egocentric Vision In Cultural Sitesmentioning

confidence: 99%

An Unsupervised Domain Adaptation Scheme for Single-Stage Artwork Recognition in Cultural Sites

Pasqualino¹,

Furnari²,

Signorello³

et al. 2020

Preprint

View full text Add to dashboard Cite

Recognizing artworks in a cultural site using images acquired from the user's point of view (First Person Vision) allows to build interesting applications for both the visitors and the site managers. However, current object detection algorithms working in fully supervised settings need to be trained with large quantities of labeled data, whose collection requires a lot of times and high costs in order to achieve good performance. Using synthetic data generated from the 3D model of the cultural site to train the algorithms can reduce these costs. On the other hand, when these models are tested with real images, a significant drop in performance is observed due to the differences between real and synthetic images. In this study we consider the problem of Unsupervised Domain Adaptation for object detection in cultural sites. To address this problem, we created a new dataset containing both synthetic and real images of 16 different artworks. We hence investigated different domain adaptation techniques based on one-stage and two-stage object detector, image-to-image translation and feature alignment. Based on the observation that single-stage detectors are more robust to the domain shift in the considered settings, we proposed a new method based on RetinaNet and feature alignment that we called DA-RetinaNet. The proposed approach achieves better results than compared methods. To support research in this field we release the dataset at the following link https://iplab.dmi.unict.it/EGO-CH-OBJ-UDA/ and the code of the proposed architecture at https://github.com/fpv-iplab/DA-RetinaNet.

show abstract

“…e system makes use of a Convolutional Neural Network (CNN) to perform object classi cation and artwork classi cation. In [17] is discussed an approach for egocentric image classi cation and object detection based on Fully Convolutional Networks (FCN). e system is adapted to mobile devices to implement an augmented audio-guide.…”

Section: Related Workmentioning

confidence: 99%

Egocentric Visitors Localization in Cultural Sites

Ragusa

Furnari

Battiato

et al. 2019

J. Comput. Cult. Herit.

View full text Add to dashboard Cite

We consider the problem of localizing visitors in a cultural site from egocentric ( rst person) images. Localization information can be useful both to assist the user during his visit (e.g., by suggesting where to go and what to see next) and to provide behavioral information to the manager of the cultural site (e.g., how much time has been spent by visitors at a given location? What has been liked most?). To tackle the problem, we collected a large dataset of egocentric videos using two cameras: a head-mounted HoloLens device and a chest-mounted GoPro. Each frame has been labeled according to the location of the visitor and to what he was looking at. e dataset is freely available in order to encourage research in this domain. e dataset is complemented with baseline experiments performed considering a state-of-the-art method for location-based temporal segmentation of egocentric videos. Experiments show that compelling results can be achieved to extract useful information for both the visitor and the site-manager.

show abstract

Fully Convolutional Network and Region Proposal for Instance Identification with Egocentric Vision

Cited by 11 publications

References 25 publications

Detecting Proximity with Bluetooth Low Energy Beacons for Cultural Heritage

Detecting Proximity with Bluetooth Low Energy Beacons for Cultural Heritage

An Unsupervised Domain Adaptation Scheme for Single-Stage Artwork Recognition in Cultural Sites

Egocentric Visitors Localization in Cultural Sites

Contact Info

Product

Resources

About