BIDAL-HCMUS@LSC2020: An Interactive Multimodal Lifelog Retrieval with Query-to-Sample Attention-based Search Engine

Mai-Nguyen, Anh-Vu; Phan, Trong-Dat; Vo, Anh-Khoa; Tran, Van-Luon; Dao, Minh-Son; Zettsu, Koji

doi:10.1145/3379172.3391722

Cited by 5 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another common approach is to transform images metadata and queries into the same space through an embedding algorithm to be able to make the comparison. This idea was adopted by almost the half of the participants in the LSC'20 [17,20,26,27,39]. On the other hand, SOMHunter [27] which was the runner-up system of the competition and Exquisitor [17] heavily relied on user's relevance feedback, while LifeGraph [33] explored the potential of indexing the lifelog into a graph structure [30].…”

Section: Related Workmentioning

confidence: 99%

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Tran

Nguyen

Bình

et al. 2021

Proceedings of the 4th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

Building an interactive retrieval system for lifelogging contains many challenges due to massive multi-modal personal data besides the requirement of accuracy and rapid response for such a tool. The Lifelog Search Challenge (LSC) is the international lifelog retrieval competition that inspires researchers to develop their systems to cope with the challenges and evaluates the effectiveness of their solutions. In this paper, we upgrade our previous Myscéal and present Myscéal 2.0 system for the LSC'21 with the improved features inspired by the novice users experiments. The experiments show that a novice user achieved more than half of the expert score on average. To mitigate the gap of them, some potential enhancements were identified and integrated to the enhanced version. CCS CONCEPTS• Information systems → Information retrieval; • Humancentered computing → Human computer interaction (HCI); User interface design.

show abstract

Section: Related Workmentioning

confidence: 99%

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Tran

Nguyen

Bình

et al. 2021

Proceedings of the 4th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

show abstract

“…Enhanced versions of LIFER, such as LIFER 2.0 [107] and LIFER 3.0 [108], were proposed with additional visual features to solve several tasks of ImageCLEFlifelog 2019 and 2020, respectively. It should be noted that many other applications have been proposed in the challenges and workshops mentioned previously [109][110][111][112][113][114].…”

Section: Lifelog Retrieval Software Toolsmentioning

confidence: 99%

Lifelog Retrieval From Daily Digital Data: Narrative Review (Preprint)

Ribeiro¹,

Trifan²,

Neves³

2021

Preprint

View full text Add to dashboard Cite

BACKGROUND The wide availability and small size, together with the decrease in pricing of different types of sensors, has made it possible, over the last decade, to acquire a huge amount of data about a person's life in real time. These sensors can be incorporated into personal electronic devices available at reasonable cost, such as smartphones and small wearable devices. They allow the acquisition of images, audio, location, physical activity and physiological signals, among other data. With these data, usually denoted as lifelog data, we can then analyze and understand personal experiences and behaviors. This process is called lifelogging. OBJECTIVE The goal of this article is to review the literature in the research area of lifelogging over the past decade and provide an historical overview on this research topic. To this purpose, we analyze lifelogging applications that monitor and assist people with memory problems. METHODS We follow a narrative review methodology to conduct a comprehensive search of relevant publications in Google Scholar and Scopus databases. In order to find these relevant publication, topic-related keywords were identified and combined based on different lifelogging type of data and applications. RESULTS A total of 124 publications were selected and included in this narrative review. 411 publications were retrieved and screened from the two scholar databases. Out of these, 114 publications were fully reviewed. In addition, 32 more publications were manually included based on our bibliographical knowledge in this research field. CONCLUSIONS The use of personal lifelogs can be beneficial to improve the life quality of people suffering from memory problems, such as dementia. Through the acquisition and analysis of lifelog data, lifelogging systems can create digital memories to be used as surrogate memory. Through this narrative we understand that contextual information can be extracted from the lifelogs and it provides significant information for understanding the daily life of people suffering from memory issues based on events, experiences and behaviors.

show abstract

“…LifeXplore is a video exploration and retrieval tool [13] combining feature map browsing, concept search and filtering, as well as hand-drawn sketching. Mai-Nguyen et al [15] bring a interactive multimodal search engine with Query-to-Sample attention-based retrieval with the aim of enhancing query formulation using textto-image mapping. Che et al [2] integrate relationship-mapping between objects and subjects in the lifelog images.…”

Section: Participating Systemsmentioning

confidence: 99%

Introduction to the Third Annual Lifelog Search Challenge (LSC'20)

Gurrin

Jónsson

Schöffmann

et al. 2020

Proceedings of the 2020 International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

The Lifelog Search Challenge (LSC) is an annual comparative benchmarking activity for comparing approaches to interactive retrieval from multi-modal lifelogs. LSC'20, the third such challenge, attracts fourteen participants with their interactive lifelog retrieval systems. These systems are comparatively evaluated in front of a live-audience at the LSC workshop at ACM ICMR'20 in Dublin, Ireland. This overview motivates the challenge, presents the dataset and system configuration used in the challenge, and briefly presents the participating teams. CCS CONCEPTS • Human-centered computing → Empirical studies in interaction design; • Information systems → Mobile information processing systems; Search interfaces.

show abstract

BIDAL-HCMUS@LSC2020: An Interactive Multimodal Lifelog Retrieval with Query-to-Sample Attention-based Search Engine

Cited by 5 publications

References 10 publications

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21

Lifelog Retrieval From Daily Digital Data: Narrative Review (Preprint)

Introduction to the Third Annual Lifelog Search Challenge (LSC'20)

Contact Info

Product

Resources

About