Broadcast news story segmentation using social network analysis and hidden markov models

Vinciarelli, Alessandro; Favre, Sarah

doi:10.1145/1291233.1291287

Cited by 32 publications

(19 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By identifying the occurrence of two anchormen and the end of first anchorman's talk, this work reported promising results in segmenting two parts of programs. They further extended their work to perform generic news story segmentation [23]. Similarly, they recognize each actor as in a "story" or as an "anchorman" from audio information, and then segment news programs into stories.…”

Section: Related Workmentioning

confidence: 99%

“…As compared to the works in [23], the novelty of our work is twofold: 1) more elaborate graph-based analysis and 2) story segmentation based on role's social context. Both works represent social relationship as a graph.…”

Section: Related Workmentioning

confidence: 99%

“…The results of story segmentation are presented in terms of purity [23]. Given the ground truth of stories and the results of story segmentation , the purity is defined as (15) where is the length of overlap between the stories and is the length of the story , and is the total length of all stories.…”

Section: B Story Segmentationmentioning

confidence: 99%

See 2 more Smart Citations

RoleNet: Movie Analysis from the Perspective of Social Networks

Weng

Chu

2009

IEEE Trans. Multimedia

149

129

View full text Add to dashboard Cite

Abstract-With the idea of social network analysis, we propose a novel way to analyze movie videos from the perspective of social relationships rather than audiovisual features. To appropriately describe role's relationships in movies, we devise a method to quantify relations and construct role's social networks, called RoleNet. Based on RoleNet, we are able to perform semantic analysis that goes beyond conventional feature-based approaches. In this work, social relations between roles are used to be the context information of video scenes, and leading roles and the corresponding communities can be automatically determined. The results of community identification provide new alternatives in media management and browsing. Moreover, by describing video scenes with role's context, social-relation-based story segmentation method is developed to pave a new way for this widely-studied topic. Experimental results show the effectiveness of leading role determination and community identification. We also demonstrate that the social-based story segmentation approach works much better than the conventional tempo-based method. Finally, we give extensive discussions and state that the proposed ideas provide insights into context-based video analysis.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

RoleNet: Movie Analysis from the Perspective of Social Networks

Weng

Chu

2009

IEEE Trans. Multimedia

149

129

View full text Add to dashboard Cite

show abstract

“…, a G } (G is the total number of speakers in the conversation and the a i are the speaker labels). Even if such an information is relatively basic and it seems to miss the richness of a conversation, still it allows one to capture a wide range of social phenomena such as the groups forming around discussion topics [13], the fronts opposing one another in competitive discussions [12], dominant individuals [8], etc. The rest of this section shows how the same information can be used to infer the roles in several interaction settings.…”

Section: Prosody: Spotting Journalists In Broadcast Datamentioning

confidence: 99%

More Than Words: Inference of Socially Relevant Information from Nonverbal Vocal Cues in Speech

Vinciarelli

Salamin

Mohammadi

et al. 2011

Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces. Theoretical and Practical Issues

Self Cite

View full text Add to dashboard Cite

Abstract. This paper presents two examples of how nonverbal communication can be automatically detected and interpreted in terms of social phenomena. In particular, the presented approaches use simple prosodic features to distinguish between journalists and non-journalists in media, and extract social networks from turn-taking to recognize roles in different interaction settings (broadcast data and meetings). Furthermore, the article outlines some of the most interesting perspectives in this line of research.

show abstract

“…This segmentation gets refined as we apply the learning algorithm, and possibly involve the user in the loop. This windowing approach has been used in the segmentation of audio broadcast news into stories [18]. In the absence of a numeric way to define data tokens, these windows provide a simple way to define a feature vector.…”

Section: Theorymentioning

confidence: 99%

A visual analytics approach to model learning

Garg

Ramakrishnan

Mueller

2010

2010 IEEE Symposium on Visual Analytics Science and Technology

View full text Add to dashboard Cite

The process of learning models from raw data typically requires a substantial amount of user input during the model initialization phase. We present an assistive visualization system which greatly reduces the load on the users and makes the process of model initialization and refinement more efficient, problem-driven, and engaging. Utilizing a sequence segmentation task with a Hidden Markov Model as an example, we assign each token in the sequence a feature vector based on its various properties within the sequence. These vectors are then clustered according to similarity, generating a layout of the individual tokens in form of a node link diagram where the length of the links is determined by the feature vector similarity. Users may then tune the weights of the feature vector components to improve the segmentation, which is visualized as a better separation of the clusters. Also, as individual clusters represent different classes, the user can now work at the cluster level to define token classes, instead of labelling one entry at time. Inconsistent entries visually identify themselves by locating at the periphery of clusters, and the user then helps refine the model by resolving these inconsistencies. Our system therefore makes efficient use of the knowledge of its users, only requesting user assistance for non-trivial data items. It so allows users to visually analyze data at a higher, more abstract level, improving scalability. INTRODUCTIONWith the tremendous growth in physical and online data collection technology, we are now experiencing an explosion of digital information. Since a large amount of these data are unstructured, various machine learning techniques have been developed to assign structure to these data to make them machine readable. This process can allow the machine to reason with and draw insight from data almost automatically. However, all such tasks depend heavily on large amounts of user-tagged data as the starting point, and use various semi-supervised learning methods [19]. Due to the high user input required, such tagged data is difficult to construct. Further, data is dynamic, and as a dataset grows and changes, we might need to supplement the tagged data from time to time. We propose to make this task simpler and interactive by designing a system where the user can obtain a visual overview of the dataset, and in that visual interface only tags those data elements that the system cannot easily resolve itself.One crucial idea behind our system is that given good feature vectors to represent each data point, points that are similar will be close-by in the feature vector space. Here, we mean data-points which though rich in semantics, do not have an explicit highdimensional feature vector automatically attached to them. In such cases we need to design feature vectors to represent the semantics and structure of the data-points. We aim to achieve this in our system by designing feature vectors which encompass a data point's structure, context, and location in the dataset. If some s...

show abstract

Broadcast news story segmentation using social network analysis and hidden markov models

Cited by 32 publications

References 5 publications

RoleNet: Movie Analysis from the Perspective of Social Networks

RoleNet: Movie Analysis from the Perspective of Social Networks

More Than Words: Inference of Socially Relevant Information from Nonverbal Vocal Cues in Speech

A visual analytics approach to model learning

Contact Info

Product

Resources

About