Multimodal joint learning for personal knowledge base construction from Twitter-based lifelogs

Yen, An-Zi; Huang, Hen-Hsen; Chen, Hsin-Hsi

doi:10.1016/j.ipm.2019.102148

Cited by 13 publications

(14 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Balog and Kenter [1] define the concept of personal KB asła resource of structured information about entities personally related to its usež, which basically covers the major life events. The general life events extraction from social media posts to construct personal KB [45,46] is also proposed.…”

Section: From World Kb To Personal Kbmentioning

confidence: 99%

“…Although these works achieve good performance on detecting personal life events, the major life events are not enough to support the personal KB construction. To this end, we propose a joint learning model [45,46] to extract general life events from social media posts.…”

Section: Textual Datamentioning

confidence: 99%

“…Recently, the public social media posts and blog posts are utilized to explore models for personal KB construction [45,46], image recall [9], and information recall [47]. The innovative strategies are conducted on lifelogging applications with the data that users are willing to share.…”

Section: Ethical and Privacy Issuesmentioning

confidence: 99%

See 2 more Smart Citations

Ten Questions in Lifelog Mining and Information Recall

Yen

Huang

Chen

2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

With the advance of science and technology, people are used to recording their daily life events via writing blogs, uploading social media posts, taking photos, or filming videos. Such rich repository personal information is useful for supporting human living assistance, such as information recall service. The main challenges are how to store and manage personal knowledge from various sources, and how to provide support for people who may have difficulty recalling past experiences. In this position paper, we propose a research agenda on personal knowledge mining from various sources of lifelogs, personal knowledge base construction, and information recall for assisting people to recall their experiences. Ten research questions are formulated.

show abstract

Section: From World Kb To Personal Kbmentioning

confidence: 99%

Section: Textual Datamentioning

confidence: 99%

See 1 more Smart Citation

Ten Questions in Lifelog Mining and Information Recall

Yen

Huang

Chen

2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

show abstract

“…In contrast, this type of research applied subjective information of users as an object to study its relevance characteristics. For example, Yen, Huang, and Chen (2019) built up a user's knowledge base relied on their daily tweets. Other authors focused on the user's loneliness on Twitter, which discussed the association features of tweets containing the keyword "loneliness" (Mahoney et al, 2019).…”

Section: Topic Analysis For Smmentioning

confidence: 99%

Interactive Evolution of Multidimensional Information in Social Media for Public Emergency: A Perspective from Optics Scattering

Xiao

2021

Data and Information Management

View full text Add to dashboard Cite

Most of the current research on the information analysis of social media (SM) for public emergency focused on a single dimension such as emotion while neglecting the interaction between multidimensional information. Therefore, in this study, an information dispersing–superimposing model is proposed to explain the implicit regularity of the impact within a symbol, sentiment, and context information and their dependent evolution on the SM. Information hue, saturation, and flux (HSF) are defined to measure the interaction process. An online event was selected to verify the concept and hypothesis of this study. The results proved that the interaction among multidimensional information did exist on the SM for a public emergency. The turning points of information dispersing–superimposing often emerged when the number of online users involved had significant changes, and sentiment and context information were showed to have a strong interaction relationship and tended to be spread at the same time. It was also manifested that the dominant information component was varied at each stage of the emergency. This paper is one of the first to study the interaction of multidimensional information on the SM derived from optics scattering. The findings of the study will try to provide a theoretical explanation for why certain information components may be enhanced during the online dissemination and suggest practical support for the information predictions and interface design for SM.

show abstract

“…To sum up, based on the the categories of lifelog activities defined in LiveKB [23,24], visual lifelog activity recognition dataset [11], NTCIR lifelog dataset [12,14], ImageCLEF lifeLog dataset [5], and Lifelog Search Challenge dataset [13], the lifelog activities in VisLife can be divided into seven life event types shown as follows.…”

Section: Lifelog Dataset Constructionmentioning

confidence: 99%

Personal Knowledge Base Construction from Multimodal Data

Yen

Chang

Huang

et al. 2021

Proceedings of the 2021 International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

With the passage of time, people often have misty memories of their past experiences. Information recall support for people by collecting personal lifelogs is emerging. Recently, people tend to record their daily life via filming Video Weblog (VLog), which contains visual and audio data. These large scale multimodal data can be used to support information recall service that enables users to query their past experiences. The challenging issue is the semantic gap between the visual concept and the textual query. In this paper, we aim to extract personal life events from vlogs shared on YouTube and construct a personal knowledge base (PKB) for individuals. A multitask learning model is proposed to extract the components of personal life events, such as subjects, predicates and objects. The evaluation is performed on a video collection from three YouTubers who are English native speakers. Experimental results show our model achieves promising performance.

show abstract

Multimodal joint learning for personal knowledge base construction from Twitter-based lifelogs

Cited by 13 publications

References 10 publications

Ten Questions in Lifelog Mining and Information Recall

Ten Questions in Lifelog Mining and Information Recall

Interactive Evolution of Multidimensional Information in Social Media for Public Emergency: A Perspective from Optics Scattering

Personal Knowledge Base Construction from Multimodal Data

Contact Info

Product

Resources

About