Nyström Approximated Temporally Constrained Multisimilarity Spectral Clustering Approach for Movie Scene Detection

Panda, Rameswar; Kuanar, Sanjay K.; Chowdhury, Ananda S.

doi:10.1109/tcyb.2017.2657692

Cited by 22 publications

(6 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…review including, for example, transcript-based approaches, news segmentation, and scene retrieval, see [8]. A prevalent approach for scene detection is to perform a variety of clustering techniques [2,4,21]. By representing video shots in some feature space the assumption is that shots from the same scene will cluster together.…”

Section: Previous Workmentioning

confidence: 99%

Learnable Optimal Sequential Grouping for Video Scene Detection

Rotman,

Yaroker,

Amrani

et al. 2022

Preprint

View full text Add to dashboard Cite

Video scene detection is the task of dividing videos into temporal semantic chapters. This is an important preliminary step before attempting to analyze heterogeneous video content. Recently, Optimal Sequential Grouping (OSG) was proposed as a powerful unsupervised solution to solve a formulation of the video scene detection problem. In this work, we extend the capabilities of OSG to the learning regime. By giving the capability to both learn from examples and leverage a robust optimization formulation, we can boost performance and enhance the versatility of the technology. We present a comprehensive analysis of incorporating OSG into deep learning neural networks under various configurations. These configurations include learning an embedding in a straight-forward manner, a tailored loss designed to guide the solution of OSG, and an integrated model where the learning is performed through the OSG pipeline. With thorough evaluation and analysis, we assess the benefits and behavior of the various configurations, and show that our learnable OSG approach exhibits desirable behavior and enhanced performance compared to the state of the art.

show abstract

Section: Previous Workmentioning

confidence: 99%

Learnable Optimal Sequential Grouping for Video Scene Detection

Rotman,

Yaroker,

Amrani

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Spectral clustering methods are closely related to this paper and were successfully applied in segmentation [12], [16]; semisupervised learning [24]; multitask learning [25]; scene detection [26]; and so on [27]- [29]. Representative spectral clustering methods include Min Cut [10], Rcut [11], Ncut [12], and Min-Max Cut [13].…”

Section: Related Workmentioning

confidence: 99%

Spectral Clustering by Joint Spectral Embedding and Spectral Rotation

Pang

Xie

Nie

et al. 2020

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Spectral clustering is an important clustering method widely used for pattern recognition and image segmentation. Classical spectral clustering algorithms consist of two separate stages: 1) solving a relaxed continuous optimization problem to obtain a real matrix followed by 2) applying K-means or spectral rotation to round the real matrix (i.e., continuous clustering result) into a binary matrix called the cluster indicator matrix. Such a separate scheme is not guaranteed to achieve jointly optimal result because of the loss of useful information. To obtain a better clustering result, in this paper, we propose a joint model to simultaneously compute the optimal real matrix and binary matrix. The existing joint model adopts an orthonormal real matrix to approximate the orthogonal but nonorthonormal cluster indicator matrix. It is noted that only in a very special case (i.e., all clusters have the same number of samples), the cluster indicator matrix is an orthonormal matrix multiplied by a real number. The error of approximating a nonorthonormal matrix is inevitably large. To overcome the drawback, we propose replacing the nonorthonormal cluster indicator matrix with a scaled cluster indicator matrix which is an orthonormal matrix. Our method is capable of obtaining better performance because it is easy to minimize the difference between two orthonormal matrices. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed method (called JSESR).

show abstract

“…A prevalent approach for scene detection is to perform a variety of clustering techniques [2,4,21]. By representing video shots in some feature space the assumption is that shots from the same scene will cluster together.…”

Section: Video Scene Detectionmentioning

confidence: 99%

Learnable Optimal Sequential Grouping for Video Scene Detection

Rotman

Yaroker

Amrani

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Video scene detection is the task of dividing videos into temporal semantic chapters. This is an important preliminary step before attempting to analyze heterogeneous video content. Recently, Optimal Sequential Grouping (OSG) was proposed as a powerful unsupervised solution to solve a formulation of the video scene detection problem. In this work, we extend the capabilities of OSG to the learning regime. By giving the capability to both learn from examples and leverage a robust optimization formulation, we can boost performance and enhance the versatility of the technology. We present a comprehensive analysis of incorporating OSG into deep learning neural networks under various configurations. These configurations include learning an embedding in a straightforward manner, a tailored loss designed to guide the solution of OSG, and an integrated model where the learning is performed through the OSG pipeline. With thorough evaluation and analysis, we assess the benefits and behavior of the various configurations, and show that our learnable OSG approach exhibits desirable behavior and enhanced performance compared to the state of the art. CCS CONCEPTS • Computing methodologies → Video segmentation; Supervised learning; Temporal reasoning; Visual content-based indexing and retrieval; • Information systems → Video search; • Theory of computation → Dynamic programming.

show abstract

Nyström Approximated Temporally Constrained Multisimilarity Spectral Clustering Approach for Movie Scene Detection

Cited by 22 publications

References 53 publications

Learnable Optimal Sequential Grouping for Video Scene Detection

Learnable Optimal Sequential Grouping for Video Scene Detection

Spectral Clustering by Joint Spectral Embedding and Spectral Rotation

Learnable Optimal Sequential Grouping for Video Scene Detection

Contact Info

Product

Resources

About