Leveraging multimodal information for event summarization and concept-level sentiment analysis

Shah, Rajiv Ratn; Yu, Yi; Verma, Akshay; Tang, Suhua; Shaikh, Anwar Dilawar; Zimmermann, Roger

doi:10.1016/j.knosys.2016.05.022

Cited by 68 publications

(11 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comprehensive reviews on personalized tag recommendation and ranking can be found in a textbook [40]. In this way, multimodal analysis [32,33] using various types of features has improved the performance of applications such as recommendation and ranking [34,37,39,40] as well as basic theories including temporal segmentation [36] and event recognition [35,38].…”

Section: Related Workmentioning

confidence: 99%

Sentiment-aware personalized tweet recommendation through multimodal FFM

Harakawa

Takehara

Ogawa

et al. 2018

Multimed Tools Appl

View full text Add to dashboard Cite

For realizing quick and accurate access to desired information and effective advertisements or election campaigns, personalized tweet recommendation is highly demanded. Since multimedia contents including tweets are tools for users to convey their sentiment, users' interest in tweets is strongly influenced by sentiment factors. Therefore, successful personalized tweet recommendation can be realized if sentiment in tweets can be estimated. However, sentiment factors were not taken into account in previous works and the performance of previous methods may be limited. To overcome the limitation, a method for sentiment-aware personalized tweet recommendation through multimodal Field-aware Factorization Machines (FFM) is newly proposed in this paper. Successful personalized tweet recommendation becomes feasible through the following three contributions: (i) sentiment factors are newly introduced into personalized tweet recommendation, (ii) users' interest is modeled by deriving multimodal FFM that enables collaborative use of multiple factors in a tweet, i.e., publisher, topic and sentiment factors, and (iii) the effectiveness of using sentiment factors as well as publisher and topic factors is clarified from results of experiments using real-world datasets related to worldwide hot topics, "#trump", "#hillaryclinton" and "#ladygaga". In addition to showing the effectiveness of the proposed method, the applicability of the proposed method to other tasks such as advertisement and social analysis is discussed as a conclusion and future work of this paper.

show abstract

Section: Related Workmentioning

confidence: 99%

Sentiment-aware personalized tweet recommendation through multimodal FFM

Harakawa

Takehara

Ogawa

et al. 2018

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…This approach is compared with baseline algorithms that determine positive, negative and neutral opinion. It is a real and complex problem in order to extract noise and maintain the semantic and sentics from media, leveraging multi-model framework [7]. The proposed approach focuses on event summarization and concept-level sentimental analysis.…”

Section: Dictionary Based Approachmentioning

confidence: 99%

Perspectives of the performance metrics in lexicon and hybrid based approaches: a review

Rani¹,

Sumathy²

2017

IJET

View full text Add to dashboard Cite

Online social media and social networking services experience a drastic development in the present scenario. Contents generated by hundreds of millions of users are used for communication in general. Users mark their opinion and review in various applications such as Twitter, Facebook, YouTube, Weibo, Flicker, LinkedIn, Online-e commerce sites, Microblogging sites, etc. User generated text is spread rapidly on the web, and it has become tedious to analyze the opinionated text in order to arrive at a decision. Sentiment analysis, a subcategory of text mining is the major active research domain in current era due to greater quantity of opinionated text present in the Internet. Semantic detection is the sub-class in the sentiment analysis which is used for measuring the sentiment orientation in any text. Opinionated text is used for analyzing and making the decision simple. This interdisciplinary field draws various techniques from data mining, machine learning, natural language processing, lexicon based and hybrid based approaches. This paper provides a broad perspective with the highlight of the current state-of art techniques emphasizing the various research challenges and gaps present. The performance metrics in terms of detection rate, precision, recall, f-measure/score, average mean, auto-Pearson correlation, cosine similarity and ratio of time on various algorithms is discussed in detail. An analysis of the text mining approaches in different domains is presented.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Researches in a single-modal task include, e.g., using deep networks for scene recognition [4], using deeper networks to achieve better performance in image classification [5], [6], generalizing features extracted from a specific dataset in a fully supervised fashion for generic tasks [7]. Some works investigated multiple modalities, e.g., event summarization by leveraging both visual and textual information [8]. Recently, there are some initial work on cross-modal correlation, such as predicting answers given an image and a question as input [9], analyzing pairwise correlation between images and their captions [10], cross-modal retrieval by canonical correlation analysis (CCA) [11] or kernel CCA (KCCA) [12].…”

Section: Introductionmentioning

confidence: 99%

Category-Based Deep CCA for Fine-Grained Venue Discovery From Multimodal Data

Tang

Aizawa

et al. 2019

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

In this work, travel destinations and business locations are taken as venues. Discovering a venue by a photograph is very important for visual context-aware applications. Unfortunately, few efforts paid attention to complicated real images such as venue photographs generated by users. Our goal is fine-grained venue discovery from heterogeneous social multimodal data. To this end, we propose a novel deep learning model, category-based deep canonical correlation analysis. Given a photograph as input, this model performs: 1) exact venue search (find the venue where the photograph was taken) and 2) group venue search (find relevant venues that have the same category as the photograph), by the cross-modal correlation between the input photograph and textual description of venues. In this model, data in different modalities are projected to a same space via deep networks. Pairwise correlation (between different modality data from the same venue) for exact venue search and category-based correlation (between different modality data from different venues with the same category) for group venue search are jointly optimized. Because a photograph cannot fully reflect rich text description of a venue, the number of photographs per venue in the training phase is increased to capture more aspects of a venue. We build a new venue-aware multimodal data set by integrating Wikipedia featured articles and Foursquare venue photographs. Experimental results on this data set confirm the feasibility of the proposed method. Moreover, the evaluation over another publicly available data set confirms that the proposed method outperforms state of the arts for cross-modal retrieval between image and text.

show abstract

Leveraging multimodal information for event summarization and concept-level sentiment analysis

Cited by 68 publications

References 9 publications

Sentiment-aware personalized tweet recommendation through multimodal FFM

Sentiment-aware personalized tweet recommendation through multimodal FFM

Perspectives of the performance metrics in lexicon and hybrid based approaches: a review

Category-Based Deep CCA for Fine-Grained Venue Discovery From Multimodal Data

Contact Info

Product

Resources

About